Introduction

• After looking at the empirical data on the matter of frauds, it was concluded that avoiding the 5% of firms with the highest ex ante predicted fraud risk would allow an investor to avoid 29% of fraud cases and over 40% of the total dollar losses from fraud.
• Another important finding was that there was no evidence that investors received compensation for fraud risk through superior performance or lower fees.
• The Bernie Madoff case, along with the many other smaller cases, have emphasized, time and time again, the opportunities advisers have to exploit investors and the importance of limiting advisers’ opportunistic behavior through either market or regulatory forces.
• In the U.S., the regulatory system protects investors primarily through mandatory disclosures. Investment advisers must file 𝐹𝑜𝑟𝑚 𝐴𝐷𝑉 to disclose information about their –
  • Operations
  • Conflicts of interest
  • Disciplinary histories
  • Other material facts
• To address the question of whether investors can use these mandatory disclosures to predict fraud, lets use an annual panel of 𝐹𝑜𝑟𝑚 𝐴𝐷𝑉s filed from August 2001 through July 2006.
• The panel includes 13,853 investment advisers who advise more than 20 million clients and control more than $32 trillion in assets (as of August 2005). These firms advise nearly all mutual funds, nearly all institutional investment funds, and many hedge funds in the U.S.
• The data also included a review of all SEC administrative proceedings and litigation releases from August 2001 through July 2010 to identify those cases in which investment advisers defrauded their clients.
• Form ADV disclosures related to past regulatory violations, conflicts of interest, and monitoring are all significant predictors of fraud. Out-of-sample tests confirm the robustness of the fraud predictions.
• These findings are subject to several limitations. These include the following –
  1. Only detected fraud cases are included in the prediction models.
  2. Although it was found that certain characteristics, such as conflicts of interest, can predict fraud, one cannot infer that conflicts of interest cause fraud, or that their prohibition would deter fraud.
  3. In addition to the disclosures mandated by the SEC, investors may assess fraud risk using other sources of information that were not included in the models.
  4. Prediction is not the sole purpose of disclosure; it is also intended to deter fraud.
• If the 𝐹𝑜𝑟𝑚 𝐴𝐷𝑉 data was not useful for predicting fraud, then either disclosure deters fraud so effectively that it eliminates the predictability that would occur in the absence of disclosure or the disclosed information is worthless.
• Thus, the tests provide evidence that regulators require investment advisers to disclose relevant information. The predictability of fraud raises the question – “why do investors allocate money to firms with high fraud risk?”
• One possibility is that the characteristics, that predict fraud, provide offsetting benefits for investors. For example, affiliation with a brokerage firm could reduce transaction costs or expedite trading. In-house custody of clients’ assets could increase fraud risk but reduce costs, resulting in lower fees for investors.
• Many experts argue that if investors differ in their valuation of fraud risk, then some investors would accept a high level of fraud risk in return for superior performance or lower fees, while other investors would choose low fraud risk and accept worse performance or higher fees.
• This analysis, however, points to the conclusion that for all three types of funds (mentioned above), there is no evidence that investors receive compensation for fraud risk through superior performance or lower fees.
• That said, one cannot rule out the possibility that investors receive some other form of compensation. Given the surprising result that fraud risk is both predictable and apparently uncompensated, one must turn to another possibility – perhaps barriers to implementing a predictive model cause the costs to outweigh the potential benefits.

Acquiring & Utilising data Related To Investment Fraud

• The study of data related to investment fraud combines two types of data:
  1. Investment fraud data
  2. Disclosures made by investment advisers in their Form 𝐴𝐷𝑉 filings.
• To obtain investment fraud data, one must search all SEC administrative proceedings and litigation releases that contain terms like “fraud” and “investment adviser” filed during the time period in question.
• Next, from these documents, one must identify all cases that involve violations of the antifraud provisions in the Investment Advisers Act. Even when another agency initially detects the fraud case, the SEC launches an administrative proceeding, which must be observed.
• The main dependent variable to look for in this case are fraud cases that harm the firm’s investment clients. Thus, cases related to insider trading, short sale violations, brokerage fraud, or other crimes, unless they cause direct losses to the firm’s investment clients, must be ignored.
• It is often important to club extended legal scenario, i.e. all legal actions, associated with a single underlying fraud into a single ‘case’ and identify the periods in which fraud occurred.
• It is also important to note that sometimes 𝐹𝑜𝑟𝑚 𝐴𝐷𝑉 is not enough to collect all data. SEC legal filings include investment fraud cases committed by firms that did not register with the SEC, and thus were not required to file 𝐹𝑜𝑟𝑚 𝐴𝐷𝑉.
• Registered firms commit slightly over half of investment fraud cases and are responsible for the overwhelming majority of the dollar losses from fraud. Thus, the scope of the tests is limited to registered investment advisers, as these firms are responsible for the most economically meaningful fraud cases.
• Another important point to note is that the vast majority of fraud cases are firm-wide.
• It is also important to note that fraud often involves the falsification of records, some loss amounts are unavailable, and the available amounts are generally a lower bound, including only the proven losses.
• Fraud duration is defined as the period extending from the initiation of the fraud until the firm ceases the fraudulent activity. The median fraud case persists for nearly three years.
• To test whether past fraud can predict future fraud, one can search SEC administrative proceedings and litigation releases filed during the time period of the test, and create two variables –
  1. The first, Past fraud, is equal to one if a prior administrative proceeding or litigation release shows that the firm has committed fraud.
  2. The second, Past affiliated fraud, is equal to one if an affiliated firm has committed fraud (affiliation implies the firms are under common control, such as common ownership or executives).
• Both variables are restricted to include only fraud cases that harmed investment advisory clients. This restriction is consistent with the main dependent variable. Furthermore, it has been observed that firms suffer greater reputational penalties for defrauding counterparties, such as customers, than for defrauding other stakeholders.
• One must then match Past affiliated fraud to investment advisers using the affiliated firm identifiers from Schedule 𝐷 of Form 𝐴𝐷𝑉. To prevent a look-ahead bias in the predictive regressions, these variables only include fraud cases that had ceased and were publicly revealed before the time period in question.

Form ADV Data

• The Investment Advisers Act, which expressly defines and prohibits investment adviser fraud, requires all advisers with more than $25 million in assets under management and with 15 or more U.S. clients to register with the SEC.
• The Act defines an investment adviser as any entity that receives compensation for managing securities portfolios or providing advice regarding individual securities.
• Registered investment advisers must file Form 𝐴𝐷𝑉 to disclose past regulatory violations and potential conflicts of interest. Form 𝐴𝐷𝑉 contains 12 items and 4 schedules.
  • Items 1 – 6 contain descriptive information about a firm and its operations.
  • Items 7 and 8 require disclosure of certain conflicts of interest.
  • Item 9 requires disclosure regarding the custody of clients’ assets.
  • Item 10 requires disclosure of control persons.
  • Item 11 requires disclosure of past legal and regulatory violations.
  • Item 12 identifies small businesses.
  • Schedules 𝐴 – 𝐶 identify the direct and indirect owners of a firm.
  • Schedule D requires disclosure of affiliations with other financial firms.
• Until recently, investors could access the latest filings only one at a time, and past filings were unavailable. Beginning in January 2010, the SEC began to provide downloadable files of historical 𝐹𝑜𝑟𝑚 𝐴𝐷𝑉 data.
• Downloadable files from July 2006 through November 2009 contain summaries of the schedules rather than Form 𝐴𝐷𝑉’s item data.
• Downloadable files from December 2009 until the present contain the item data, but not the schedule data.

Form ADV Variables

• This table summarizes a cross- section of the investment advisers’ characteristics and disclosures, using information from each firm’s first Form 𝐴𝐷𝑉 filing during the sample.
• Panel 𝐴 shows that the median firm is wholly employee-owned. Employee ownership is included because external owners may deter fraud by monitoring employees. The Average account size is $55 million, but this variable is highly skewed and the median is only $1.4 million.

Form ADV Data – Observations

• Percent client agents is the percentage of the firm’s clients who are agents (e.g., pension fund managers) rather than the direct beneficiaries of the invested funds.
• On average, 23.2% of a firm’s clients are agents. This additional layer of agency is potentially related to fraud because agents have weaker incentives to monitor investment advisers, but may also have greater expertise and financial sophistication.
• Item 11 of 𝐹𝑜𝑟𝑚 𝐴𝐷𝑉 requires each investment adviser to disclose its disciplinary history, as well as that of its (non-clerical) employees, its affiliated firms, and the employees of affiliated firms.
• The 24 questions in Item 11 are divided into three categories: regulatory, criminal, and civil judicial. From these questions, one can create 2 indicator variables.
• Past regulatory equals one if the firm discloses past regulatory violations, indicating sanctions by the SEC, the Commodity Futures Trading Commission, or a self-regulatory organization such as the Financial Industry Regulatory Authority (FINRA).
• The second variable combines the remaining two categories. Fraud firms are significantly more likely to report both types of violations.
• The disclosure information in Item 11 covers a wide range of regulatory and legal offenses, and the offenses are often minor, such as failing to follow protocols for record storage. Minor violations seem to be the norm rather than the exception, and should be interpreted as such:
  Less than 2.5% of firms that report past violations have a prior instance of fraud.
• Form 𝐴𝐷𝑉 does not distinguish whether the investment adviser or its affiliate(s) committed the reported violations, and so there is a strong positive correlation between prior violations and the number of affiliates.
• Items 7 and 8 of Form 𝐴𝐷𝑉 require firms to disclose conflicts of interest. From this information three variables are created –
  1. Referral fees equals one if the firm compensates other parties for client referrals
  2. Interest in transaction equals one if the firm trades directly with its clients or has a direct financial interest in securities recommended to its clients.
  3. Soft dollars equals one if the firm directs clients’ trades to a brokerage with relatively high commissions and, in return, the broker supplies the adviser with research or other benefits. Since clients pay the costs while the investment adviser realizes the benefits, soft dollars create a potential conflict of interest.
• The five variables used to measure monitoring are –
  1. Broker in firm equals one if the firm employs registered representatives of a broker- dealer. Trading through an affiliated broker dealer removes one form of external oversight and provides a mechanism for fraud.
  2. Investment Company Act equals one if the firm manages money on behalf of a fund registered under the Investment Company Act, such as a mutual fund. The Act increases regulation and disallows certain conflicts of interest but also indicates the firm’s investors are relatively unsophisticated.
  3. Custody equals one if the firm has possession, or the authority to obtain possession, of its clients’ assets. Custody facilitates fraud by removing external oversight.
  4. Dedicated 𝐶𝐶𝑂 equals one if the firm’s chief compliance officer (𝐶𝐶𝑂) does not have another formal job title. All registered investment firms must designate a 𝐶𝐶𝑂 who is  responsible for ensuring compliance with SEC regulation, but often the 𝐶𝐶𝑂 has other potentially conflicting roles within the firm.
  5. Hedge fund clients equals one if over 75% of the firm’s clients are hedge fund clients. This variable is included for two reasons –
     • Hedge funds are relatively opaque, which could facilitate fraud.
     • Prior to 2006, some hedge fund advisers were not required to file Form 𝐴𝐷𝑉, which could create a sample selection bias if non-reporting is associated with fraud.
• It is important to note that most studies, must complement the Form 𝐴𝐷𝑉 data with other data. Fund level data, that is not disclosed in Form 𝐴𝐷𝑉, can be used to test the relation of fraud risk with performance and fees. For the study, fund-level data was obtained from the TASS hedge fund, CRSP mutual fund, and PSN Informa databases.
• The databases are matched to the Form 𝐴𝐷𝑉 sample using firm name, location, and assets under management

Predicting Fraud

• Lets test whether the Form 𝐴𝐷𝑉 data can predict investment fraud. The purpose of these tests is prediction and, as noted previously, no claims regarding causality are being made here.
• Many of the independent variables are endogenous (e.g., a firm’s executives may deliberately choose an organizational structure that enables fraud), but because our goal is prediction rather than establishing causality, the potential endogeneity of the independent variables does not change our interpretation.
• A major caveat in interpreting the findings is that one can observe only detected fraud. Three factors affect observed fraud:
  1. The unobservable true rate of fraud
  2. The probability of detection given a fixed level of monitoring
  3. The allocation of monitoring resources
• Ideally, the regressions will predict the true rate of fraud. However, if certain predictive variables are correlated with either monitoring or detection, this relation could affect the interpretation of the results.
• Further, the predictive variables could be correlated with monitoring and detection for two reasons-
  1. Any predictive variable that decreases the probability of detection will increase the incentive to commit fraud. In general, this problem biases against significant results because predictive variables that are associated with a higher rate of fraud will also be associated with a lower detection rate.
  2. If the difficulty of detecting fraud affects the allocation of monitoring resources, this may, or may not, outweigh the added difficulty of detecting fraud.
• The issue of undetected fraud can be addressed in two ways –
  1. Extensive out-of-sample tests need to be conducted to ensure the predictions are robust.
  2. Detected fraud cases for a longer time period, than the one being taken into consideration, would provide greater level of data.

Predicting Fraud – Prediction Models

• Some of the main variables used in predicting fraud are –
  • Past fraud
  • Past affiliated fraud
  • Past regulatory problems
  • Past civil or criminal cases
  • Referral fees
  • Interest in transaction
  • Soft dollars
  • Broker in firm
  • Investment Company Act
  • Custody
  • Dedicated CCO
  • Majority employee ownership
  • Logarithm of assets under management per client
  • Percent client agent
  • Hedge fund clients
  • Logarithm of assets under management and of firm age in years

Economic Interpretation Of Prediction Models

• The key question of interest when it comes to models used for prediction of frauds is whether the overall model would enable an investor to avoid fraud. To address this question, the predicted values from the regressions, and the tradeoff between correctly predicted fraud cases and the false positive rate, are considered.
• False positives, which occur when the model incorrectly predicts that a clean firm will commit fraud in the subsequent year, can be interpreted as the opportunity cost to investors of erroneously limiting their investment opportunity set.
• Although failing to predict fraud is likely more costly than mistakenly avoiding an honest investment adviser, an investor would need to avoid multiple honest advisers for every fraud avoided. To illustrate the possible tradeoffs between false positives and predicted fraud, this figure shows a receiver operating characteristic (𝑅𝑂𝐶) curve for the prediction model. The points on the ROC curve are generated non-parametrically by taking each observation’s predicted value from the probit model as a cut-point, and then computing both the proportion of fraud firm-years correctly predicted and the false positives.
• Random prediction of fraud would result in a straight 45 degree line. Initially, the curve rises steeply, showing a considerable number of fraud firm-years could be avoided at a low false positive rate. The 𝑅𝑂𝐶 curve, in the figure in the last slide, shows the full range of all possible tradeoffs between the prediction of fraud and false positives.

Out Of Sample Prediction Of Fraud

• A key concern for any prediction model is out-of-sample validity. To determine whether the within-sample predictions are robust out-of-sample, there are two ways –
  1. Summarizing the out-of-sample predictive performance of each model during the time period in question.
  2. Performing K-fold cross validation tests.
• To conduct an out-of-sample test, the SEC administrative proceedings and litigation releases can be studied to identify fraud cases that occurred during the time period in question.
• Further, using Form 𝐴𝐷𝑉 filings, one must assign each firm a predicted value based on the coefficients estimated within-sample. Then the test regarding whether these predicted values can accurately classify the out-of-sample fraud risk of the firms, is carried out.
• The information derived and analyzed would show the proportion of fraud cases correctly predicted at a false positive rate. The proportion of fraud cases predicted out-of-sample is usually higher than within-sample, although given the small number of observations, this difference is not statistically significant.

K-Fold Cross Validation Tests

• As a further robustness test of the predictive models, a K-fold cross-validation test, for the time period under consideration, is usually performed. The idea behind these tests is simple –
  • Each model is estimated on a randomly selected subsample of firms, and the coefficient estimates from this subsample are used to classify the firms in the holdout sample.
  • Specifically, each firm in the sample is randomly assigned to one of ten groups (note that one would randomly assign firms, and not firm-years, to avoid overstating the results due to non-independence).
  • One then estimates the prediction model ten times, excluding each randomly formed group once.
  • Each observation in the excluded group is assigned a predicted value, using the coefficients estimated from the observations in the other nine groups.
  • The cutoff scores for fraud prediction are calculated within-sample and used to classify the observations in the holdout sample.
  • One repeats this process 20 times, for a total of 200 hold-out samples. Generally, the predictive power of the models is only slightly lower in the hold-out samples.
• The 𝑅𝑂𝐶 curve shows the relation between the proportion of fraud detected and the  proportion of false positives for all possible classification cut-points. The 𝑅𝑂𝐶 curve is generated by taking each observation’s estimated fraud probability, computing the sensitivity and false positives using that point as a cut-point, and then plotting the results.
• The results of the out-of-sample and K-fold cross-validation tests together can be used to determine the robustness of the fraud predictions.

Annual Cross Sectional Regression

• Although sometimes experts use observations from the entire sample period, which allows for relatively powerful tests, this may obscure time effects, which could arise in several ways –
  • The actual rate of fraud could change over time due to changes in the legal or operating environment (e.g., poor performance could decrease the benefits of a reputation for honesty, thus increasing the incentive to commit fraud).
  • The detection rate could change over time..
• To examine whether there are time effects in the prediction of fraud, one can look at the annual, cross-sectional probit regressions that predict investment fraud that occurs during the subsequent 12 months.
• Because fraud cases can persist for multiple years, these annual regressions are not independent, and aggregating coefficients across years could lead to faulty conclusions. To test whether the coefficient estimates are significantly different across years, one can use Wald tests.
• Because the same firm can appear in multiple years, one must adjust the Wald tests for non- independence.
• It is important to note that –
  • The coefficients like Dedicated 𝐶𝐶𝑂, Custody and Majority employee ownership, can vary significantly, year on year.
  • Fewer observations in annual cross-sectional regressions, can lead to the Wald tests having low power to reject the hypothesis that the coefficients are equal across years.

Firm Wide Fraud VS Fraud By Rogue Employees

• In some cases, the executives of the firm commit or are aware of the fraud. In other cases, rogue employees evade their firms’ internal control systems.
• A potential concern with the prior tests is that rogue employee fraud is likely more frequent at firms with many employees. If the predictive variables are correlated with the number of employees, this could lead to spurious correlations with rogue employee fraud.
• To ensure that this problem does not drive the results, one must compare the predictability of firmwide and rogue employee fraud.

Are Investors Compensated For Fraud Risks ?

• Given that fraud is predictable, investors still allocate money to firms with high fraud risk. This might be because of the following -–
  • One possibility is that the characteristics associated with high fraud risk may provide offsetting benefits that improve investment performance. Some investors could voluntarily accept high fraud risk in return for higher expected returns.
  • Another possibility, is that investors pay a premium for advisers with low fraud risk, and pay lower fees to advisers with high fraud risk.
• To test whether investors are compensated for fraud risks requires fund data, which is only available for a subset of investment advisers. Hedge fund and institutional fund advisers are not required to report return data, which could create a selection bias if fraudulent advisers choose not to report returns.
• Data is available for all mutual funds and mutual fund advisers, however, and so there is no selection bias for this category of funds. One must measure each firm-year observation’s fraud risk as the predicted value from the probit regression. Further, at a certain point of time each year, adviser’s predicted fraud risk is assigned to each firm.
• Funds can be classified as high fraud risk if they are advised by a firm whose predicted fraud risk is greater than the 95^th percentile of clean firms (this is a value that analyst must use his or her discretion to come up with).
• For each of the three return database samples, one must form two equally weighted portfolios. At the beginning of the time period, one must assign all funds classified as high fraud risk to one portfolio. The remaining funds are assigned to the low fraud risk portfolio. One must then find the equally weighted returns of the portfolios for the subsequent 12 months.
• Now that there is a system in place, one can observe the level of returns on the two portfolios, both absolute return as well as the risk adjusted return. Based on empirical evidence collected studies using similar procedures as well as different ones, the general conclusion drawn is that the investors are not compensated for bearing fraud risk.

Data Access And Implementation

• Lets test how well investors could have implemented predictive models using only Form 𝐴𝐷𝑉 data that had previously been publicly accessible. These tests differ from the predictive regressions discussed previously (which use information from the full-sample period and thus, do not directly address how well an investor could have predicted fraud during the sample).
• During the sample period, the SEC did not provide public access to historical Form 𝐴𝐷𝑉 filings; investors could access only a contemporaneous cross-section. For this reason, one must compare two types of predictive models –
  1. In the first, one can estimate predictive models that use only the contemporaneous cross section of Form 𝐴𝐷𝑉 filings. These tests mimic the predictions an investor could have made during the sample period, given the actual data access policies in place.
  2. In the second, one can estimate predictive models that use data from an annual panel of historical Form 𝐴𝐷𝑉 filings. These tests mimic the predictions an investor could have made if historical filings had been publicly accessible.
• The dependent variable equals one for all firms with an observable prior fraud case (i.e., a fraud case that occurred during the time period in question). One can use the coefficient estimates  from this regression to make out-of-sample predictions of the fraud cases that occur during that time period.
• It is important to note that if the contemporaneous Form 𝐴𝐷𝑉 filings are all that is publicly accessible, then investors can only estimate fraud risk from backward-looking regressions.
• If historical filings are accessible, investors can estimate forward-looking prediction models and then estimate fraud risk by combining the estimated coefficients with the contemporaneous disclosures of the firms.
• This is a conceptually important distinction. The backward-looking regressions only include the subsample of firms that survived the legal consequences of committing fraud, which could bias the coefficients.
• However, the forward-looking models often have one disadvantage – these models require at least two years of data to estimate, which is something that can often add as a limiting factor.
• The two main issues of interest here are –
  1. First, could the prediction results had been achieved during the sample.
  2. Second, is the comparison of the predictive ability of the models – one which uses all the prior data and the one which uses only the publicly accessible data.
• Empirical evidence in various studies point to the fact that there is substantial evidence that public access to historical Form 𝐴𝐷𝑉 filings could benefit investors. Moreover, the marginal cost to the SEC of allowing public access would be quite low.
• To implement an effective fraud prediction model, an investor would have had to collect manually a large number of Form 𝐴𝐷𝑉 filings, convert the filings into a database, and estimate a prediction model. For most investors, the cost of individually downloading thousands of Form 𝐴𝐷𝑉 filings may well have exceeded the perceived benefits.
• This problem is exacerbated by the fact that investors are atomistic: Even if the aggregate benefit of processing the disclosed information outweighs the cost to a single investor, the benefit to any single investor may be insufficient.
• The socially optimal level of a crime occurs when the marginal benefit from a further reduction in the crime is equal to the marginal cost of enforcement. Allowing public access to historical Form 𝐴𝐷𝑉 filings would reduce the marginal cost of increased enforcement by facilitating investors’ use of these data.
• This, in turn, should reduce the marginal benefit to an investment adviser of committing fraud due to an increase in the probability of detection. Thus, improved public access to these disclosure data could reduce the occurrence of fraud.