[Editor’s note: This is a guest post by Bryce Mason, the founder of P2P-Picks (go here to read my review of his service). He has done some groundbreaking research here by analyzing various Lending Club investment strategies from different bloggers and calculating their real return based on historical loan data. While Bryce is obviously trying to promote his own service, I believe the analysis he has done here can benefit all investors.]
There are many opinions about P2P portfolio selection strategies, but not much analysis that allows transparent comparisons among them. At least a dozen bloggers have published strategies that aim to increase return on investment (ROI). There are also a number of pay services with proprietary strategies aimed at accredited investors (via investment funds) and regular account holders (via research subscription sites like InterestRadar and P2P-Picks). P2P investors must try to put different strategies on a level playing field and judge their value fairly, such as in the following graphic.
The purpose of this blog post is to provide a framework for individual investors to make more valid comparisons across strategies.
1 Strategies
Strategies for this post include those authored by various people around the web who have published filters—the criteria by which they select loans for investment. We followed their rules and developed lists of all LendingClub loans issued from 2008-Q3 to 2009-Q3 that each strategy would have selected, and then we simulated a $25 investment in each one. Due to the nature of proprietary strategies, we could only include our own, the P2P-Picks Profit Maximizer, and tested its three levels of recommendations: the predicted Top 1%, Top 5%, and Top 10% of all loans. So as not to use information that would have been unknowable at the time, we developed a special P2P-Picks model just for this purpose, based entirely on loans issued prior to 2008-Q3.
2 Outcomes to Consider
At least three outcomes must be considered for an investment strategy.
First is a measure of its ROI. A good measure of ROI should be based on actual performance over the entire course of the portfolio (or monthly performance of portfolios that have an average age well beyond the point at which defaults have peaked). This is important because some strategies advertise high performance on youthful portfolios, or paper over older defaults with an increasing portfolio size. Under those cases, it’s easy to hide future losses that will eventually come home to roost. Additional considerations include interpretability (such as using an annualized measure of performance) and specificity (measuring only the performance of the strategy and not confounding it with investor behavior—like allowing cash drag to eat away at returns under one strategy but not another).
For ROI we used Excel’s XIRR function, which takes a periodic set of cash flows (investments and repayments on various dates) and uses a numeric approximation method to generate an annualized return. Thankfully, we have payment dates and amounts for all LendingClub loans. This is an excellent measure of ROI for all of the reasons above. It is based on actual performance. It covers the entire investment period because loans issued at the tail end of our experiment have already matured and their outcomes are known (it was useful to wait a few additional months past maturity for late payments to trickle in). It is extremely interpretable. And it solely measures the strategy performance because only the dates of investment and repayment are considered (no cash drag is measured and no assumptions about reinvestment need be made). All ROI calculations are net of the 1% LendingClub servicing fee and, for the P2P-Picks models, the approximate 1% subscription fee.
Second is a measure of volume—or how much money can realistically be invested in a given period of time. Deploying $1,000 is easier than $1M, and different investors will have different needs. For this measure, we simply looked at the percentage of all LendingClub loans that each strategy recommended.
Third is a measure of the portfolio’s risk. Few investors consider risk explicitly, or assume that interest rates compensate for risk entirely. For the same reasons as in ROI, a good risk measure should look over the entire course of the portfolio and measure variability in the outcome. We chose to construct a simple risk metric based on each loan’s ultimate repayment amounts, and then aggregated them up to the portfolio level. More information about the definition of this metric can be found in the P2P-Picks white paper. Portfolios with more loan losses received a higher risk score.
3 Results
From the above figure, one can see that there is large variation in portfolio performance. In this case, we see that a few strategies appear to provide outsized ROI and less risk, and a few provide worse ROI and more risk than the LendingClub Index (an equal investment in every note). Once performance has been plotted in this way, preferred strategies become obvious. No strategy to the right and below the LendingClub Index would have been a worthwhile investment, as it offered lower returns, more risk, and less volume.
In terms of loan volume, the following chart demonstrates which strategies were able to offer investors a higher volume of loans, increasing the chances that in a real-world application there would be loans available for investment. Again, there is substantial variation in ROI for every level of loan availability. Any strategy below the LendingClub Index ROI line was strictly inferior.
4 Conclusion
Apples-to-apples comparisons of strategy performance can help investors make more informed decisions about how to deploy their funds. In addition to ROI, one should consider portfolio risk as well as volume produced by the strategy.
Further, a credit-model approach to selecting notes—such as P2P-Picks— appears on balance to be superior to filter-based approaches, which from the first figure can be seen to have lower returns and more risk. One reason for this is that filter-based strategies are necessarily too restrictive in terms of how they identify “good” loans. Many borrower attributes are quite fine grained and small movements in them only affect the likelihood of default a little bit. Having an absolute and arbitrary cutoff (e.g., zero inquiries) when many other borrower attributes look positive may reject an excellent investment opportunity.
The above post is a summary of Bryce’s research. You can access the complete report here.
[Update: Based on the feedback in the comments Bryce added two more quarters of data to the Risk vs Reward chart. The ROI picture has changed quite a bit – most likely because the sample size is now larger and one or two charged off loans have a less dramatic impact.]
