Margarete Bester from PIC Solutions explains credit application scoring
Application scorecards have become an integral part within consumer credit companies. In the development of these models, the length of time it takes new accounts to mature forms an important part of the process. This article examines the way maturity of new accounts is derived. The case study documents a range of industries, comparing the past maturity norms with current data driven observations. In order to present a clear picture, the scoring process will be discussed.
Most credit companies take on new business to ensure that the existing portfolio is stable and future growth occurs. Taking on this new business is considered to be risky as the business has no past understanding of the new applicants except for the application form. Credit scoring within the new accounts area is designed to examine the known demographic and socio-graphic information from application forms. The risk of new applicants could now be identified using predictive models and/or the use of bureau scores.
Scorecards are mathematical tools used to rank applicants in terms of certain qualities, the main factor being to discriminate between good and bad applicants. An applicant will typically be referred to as bad if the company, knowing the true performance, would not have accepted the applicant. Through scorecards, each applicant receives a score related to the probability of that applicant reaching the bad definition set. The accept or reject decision is then based on the level of risk the business is willing to expose themselves to. The building of an application scorecard requires the selection of a development window within which applications are selected, called an application window as well as a number of months over which the performance of the application is measured or the outcome period.
The definition of good or bad varies from company to company. Most application scorecard performance definitions are based on delinquency, namely four or more payments missed, which may represent the bad definition. There are many ways to determine the bad definition; an example is with the aid of roll rates or by determining the delinquency stage at which collections move from being a customer education or gentle action to a harsher action.
The missing part within this process of defining an applicant's status; either good or bad is the number of months over which the developer analyses the accounts. In the definition above, no accounts will be defined as bad if the accounts were only observed over three months, as even accounts that default from day one would need four months to reach the 'four payments missed' point.
It is the definition of the outcome period that is tricky. If it is too long the application data that is used could be too old and could affect the predictive power of the scorecard, whereas if it is too short, all the bads are not defined. The scorecard developer will need to determine the number of months required to show the accounts maturity. The outcome period is guided by this maturity. A portfolio is defined to be mature when applicants have had enough time to show their true performance, either paying their accounts correctly, as stipulated in the terms and conditions, or reaching a delinquent state. In scoring terms, it is defined as the number of months it takes for the majority of accounts that default to reach a specific bad definition.
Many developers use past analysis from similar markets and apply the same maturity to new developments within comparable portfolios.
X-HEAD: Development window
The importance of determining the correct maturity is best explained with the following example. An account is opened in January, paid on time each month until 14 months later it defaults, going seriously delinquent over the next few months. If this situation was similar for a large percentage of the delinquent accounts and a 12-month maturity window was selected, it would have the effect of incorrectly or not adequately providing for those accounts that go bad after 12 months, as in this example the accounts would be labelled as good or indeterminate within the modelling process. The main question now is whether the accounts that mature faster have a different make up to those that take a longer time to mature.
In determining the time to maturity, historical application data together with all performance data is used going back as far as possible. It is noted that the tests described below are not completely accurate due to the amount of data a company has archived, but it produces measurable results that are better than assuming a pre-defined or available outcome period. The methods described below are used in combination with other analysis to form a clear picture when defining the development window.
There are three different methods that can be used to calculate the outcome period,
Historical guidelines;
Summary counts;
Maturity distribution graph.
X-HEAD: Historical guidelines
Traditionally, analysts assumed a certain period to be fixed for specific types of portfolios, based on historical analysis conducted. Typically, the outcome period is assumed to be 12 months if the type of portfolio is revolving credit. The drawbacks occur where the guidelines are applied in different countries or on portfolios operating on a different market segment.
X-HEAD: Summary counts
Summary counts is a data driven summary that creates a picture of the portfolio. The report is created by analysing the changes in the percentage of bad accounts on a monthly basis, moving backwards from the most recent months.
The table below shows the volume and percentage of good and bad accounts by month. This illustration is done on fictional data just for example purposes. The month represents the date the account was opened. The volume in each cell represents the number of accounts meeting the performance definition over the next months up until the present time, in this case December 2001. The delinquency of accounts opening in September 2003 will be analysed for the months September, October, November and December. If the bad definition is reached in any of these months the account is marked as bad.
The method used to determine the maturity time, is to view the volumes and percentages of bad applicants as well as the total number of accounts for each month. Table one shows that the portfolio is affected by seasonality peaking over the January and April periods, which will also have an effect on the bad percentages. An analysis of the bad percentages determines where the accounts have stabilised. Starting at the most recent months the first few months will not show any bad accounts due to the fact that the bad definition has not been reached yet. Considering the bad definition of being bad after more than three payments were missed, the first bad accounts will be shown in month number four. Each subsequent month is reviewed until a point where a stable month is located. In the example, the volume of bad accounts seems to stabilise after eight months. This could represent the maturity. The drawback of using summary counts is that seasonality could make it difficult to determine the correct point at which the number of bad accounts stabilises.
X-HEAD: Maturity distribution graph
A more recent method developed by PIC Solutions is through the creation of a maturity distribution. Only the bad accounts are selected for this analysis using accounts that have had at least 12 months to mature. This ensures that only applicants which had a longer time to mature are taken into account and no bias is included in the analysis to ensure all the accounts have an equal chance of showing their time to mature. During the computation of the distribution, a variable containing the number of months each account took to reach the bad definition is produced. An example showing the cumulative percentage of bad accounts by the number of months it took for the accounts to become bad, may be seen in figure one.
The scorecard developer typically looks at the point the graph levels out. In this example the maturity would be approximately 20 months as indicated by the red line in figure one. If this point was selected the model would use approximately 80 per cent or 90 per cent of the available bad applicants and this would indicate that the outcome period should include at least 20 months worth of performance data per application. The type of window will also affect this final outcome period. If the window used to evaluate the performance of the applicants is a fixed time interval for each applicant it is called a normal window, otherwise if it is a window that is stretched over all the available data for each applicant, it is called an ever window.
The dilemma for a scorecard developer in selecting the development window is to ensure that applications are selected as close to the current position as possible, but at the same time with the most appropriate outcome period. This can be seen in figure two, where a longer maturity window forces the application selection further into the past. Failing to get the most recent applicants could mean that when the scorecard is implemented, it does not work effectively on the future population.
With the above information in mind the maturity distribution graph could be re-examined. A 21 to 33 month old application may not accurately reflect the profile of new applicants applying and a compromise between the two situations could result in a shorter maturity window of 18 months.
Other attributes of the specific portfolio such as the growth of the portfolio, must also be taken into consideration by the analyst when the specific window is selected. In the case of a very fast growing portfolio, a higher weight should be placed on selecting the most current application dates as possible, than on the maturity of the portfolio.
X-HEAD: The Maturity of Different Portfolios
The outcome period of an application scorecard may differ from portfolio to portfolio due to the dynamics of each, but as stated before, there are other factors that may influence the outcome, for example seasonality, the product type, dynamics and portfolio growth as well as the maturity time.
The case study examines three different portfolios
retail credit;
auto loan;
other credit products.
X-HEAD: Retail credit
Revolving credit such as retail credit or mobile phone accounts with a limit, is traditionally expected to mature within six to eight months.
With a credit or retail account, a certain credit limit is granted to an applicant and the balance is paid off in monthly instalments. Using the maturity distribution method, it was observed that these portfolios took approximately 18 and more months to mature. Therefore using the historical observations of six to eight months, implies that a large number of bad accounts would not be included in the scorecard build. In Figure three, the maturity distribution graph for a typical retail account portfolio is presented. It can be observed that only 30 per cent of the accounts show their true performance after six months, as indicated by the vertical line.
X-HEAD: Auto loan
Considering car loans where applicants are granted a loan for typically three to five years, it is traditionally expected that the applicants will show their true performance within the first nine to 12 months, but in fact only 75 per cent of the applicants matured within 12 months. Most applicants matured after 19 months. The graph in figure four, shows that the 19 or more months would give a better picture of the bads. On the other hand, scorecard developers strive to choose an application window as near as possible to the most recent date to ensure an accurate prediction of risk.
X-HEAD: Other Credit Products
Longer term mortgage loans have traditionally matured between 12 and 18 months, but it is observed in some of the portfolios to take between 19 and 24 months to mature. Using the traditional 18 months there is a possibility of loosing out on accounts going bad after 18.
The same is expected to happen with micro-lending where it is traditionally expected that a micro-lending portfolio will show its true performance within a few weeks up until one or two months, but some micro-lending portfolios take more than three months to mature which means that by following the traditional rule of thumb, the scorecard builder will lose out on a significant number of bads.
In each test conducted, a different maturity window was observed by using the maturity distribution graph, when compared to the past windows in similar portfolios. The main idea of the exercise is not to redefine the portfolio maturity window, but to encourage developers to use data driven methodologies when building scorecards. This, together with all the other analysis, will ensure a defined bad population is created and used within the modelling process.
Margarete Bester is a consultant for risk management specialists PIC Solutions
Banker Middle East 2007
