The impact of financial restatements on financial markets: a systematic review of the literature


Purpose
The purpose of this paper is to discuss the most relevant issues related to the impact of financial restatements in the dynamics of financial markets and identify several research gaps to be investigated in future research.


Design/methodology/approach
The methodology is based on a systematic review of the literature described by Tranfield et al. (2003). The final sample includes 47 academic papers published from 1996 to 2019.


Findings
Papers in this domain discuss three main topics: how the market prices the announcement of a financial restatement; how financial restatements affect the announcing firm’s cost of capital and how financial restatements affect firms’ reputation. There are several issues to explore in future research, including whether financial restatements affect the dynamics of financial markets in Europe, whether the market fully and promptly assimilates the information content of a restatement, the role of financial analysts’ information disclosures in this process or how regulators may improve the way they provide investors with timely information about firms’ restating problems.


Research limitations/implications
There is always some degree of subjectivity in the definition of the keywords, search strings and selection criteria in a systematic review. These are all important aspects, as they delimitate the scope of the study and define the sample of papers to be reviewed.


Practical implications
The answers to the research questions identified in this paper may provide regulators with information to improve financial accounting and reporting standards and strengthen investors’ confidence in accounting information and the dynamics of financial markets.


Originality/value
This paper systematically reviews the relevant literature exploring the connection between financial restatements and the dynamics of financial markets. It contributes to the academic community by identifying several research questions that may impact the theory and practice related to accounting quality and capital markets.


I would like to thank my colleagues Marisa Cesário, Dora Agapito and Emília Madeira for their interest in accompanying the course of this work, and for some words that, sometimes without knowing, gave me a boost in my motivation.
A special thanks to Professor Efigénio Rebelo who have always encouraged me to finish the dissertation, for the precious advice and the incitation to always go further. 3. Identificar e discutir as questões mais importantes resultantes da ligação destas duas áreas assim como os desenvolvimentos mais recentes; 4. Identificar as oportunidades de investigação que possibilitem a realização de trabalho empírico no futuro.
xii ABSTRACT This dissertation reviews the literature systematically regarding the impact of financial restatements on financial markets and identifies some research avenues that can be explored in future empirical work. This accounting event is a clear case of bad news and affects several market participants.
The methodology employed is the systematic review of the literature that aims at minimising the weaknesses and biases of the traditional literature review. One of the most robust conclusions is that the short-term market reaction to the disclosure of a restatement varies between 1.4% and 20%. The magnitude of the impact depends on the cause and reason for the restatement, who initiates it, if there is litigation and if there is fraud. In the long-term, it is not clear if the market fully assimilates the information contained in a financial restatement.
There is also evidence that the market anticipates the publication of a financial restatement given the significant and negative abnormal returns in the pre-event period.
Also, the short-selling activity increases in the pre-event period and during the days surrounding the disclosure of such accounting event. Together, these findings suggest that market participants can anticipate this event. Moreover, financial restatement firms experience an increase in the cost of capital, an increase in the cost of debt, a decrease in the reputation of the company and cause a contagious effect on other firms operating in the same industry.
The results of this systematic review emphasise that the market impact of financial restatements is a relevant topic in the accounting and finance domain and there are some research avenues that may be explored in further empirical work related to the longterm dynamics of financial markets and the role of some sophisticated agents in the phenomenon.
Keywords: financial markets; financial statements; fraud; irregularities; restatements. xiii xiv    High quality financial reporting has been a concern for companies and stakeholders for a long time. In the beginning of the 21 st century, after the debacles of WorldCom and should be a new standard in order to put an end to a culture of irresponsibility (Jacob and Madu, 2009).

LIST OF TABLES
This chapter presents an initial review of the literature addressing financial restatements. It starts by highlighting some relevant historical facts in the development of accounting, the first attempts to establish accounting principles to be universally used by firms' financial statements, the more recent attempts to increase investors' confidence in the corporate financial reports and the search for accounting quality and transparency in the financial reporting and financial markets. In a second stage, this chapter discusses financial restatements and identifies the reasons, motives and consequences of such accounting event. Central to the discussion is how financial markets respond to restatements. The last part of this chapter looks at the concept of ´market efficiency´ and how it relates to financial restatements.

Accounting History
The advent of World War I and the many harmful consequences forced Germany to pay for the damage caused to other Nations (Treaty of Versailles). This obligation contributed to a crisis of hyperinflation in Germany, causing the maladjustment on the values of assets/liabilities in the companies. To solve the problem of adjustment, Eugene Schmalenbach developed the dynamic theory of price adjustment based on two fundamental principles: the periodic income (as a measure of financial efficiency) and comparability (Martínez Tapia, 1995).
Schmalenbach' work changed the static vision of accounting. Accounting became dynamic and based on the recognition of assets valued at cost and then depreciated/ amortised over time. Such change had a profound impact on accounting theory in the United States during the 20th century. In fact, at the beginning of that century, accounting theory was poorly organised as it was the result of a mix of texts and treatises written by academics and accountants prescribing how financial reporting should be done. As such, there was no standardisation of the principles underlying accounting, and this is one of the reasons that may have led to the financial crisis of the late 1920s.
The enactment of the Securities and Exchange Act (SEC), which established the commission of the same name in the 1930s and the creation of the American Institute of Certified Public Accountants (AICPA) were the first serious steps towards establishing a set of accounting principles and a 'Conceptual Structure' that could be universally used by US firms (Baker, 2017 Despite all the developments in the accounting thinking, "accounting theorists agree that no comprehensive theory of accounting has yet been developed" (Coetsee, 2010).
Importantly, the accounting standards set by the FASB are the 'generally accepted accounting principles' (GAAP). More recently, the scandals occurred in the beginning of the 21 st century led the U.S. Congress to approve the Sarbanes-Oxley Act (SOX) to regain investors' trust. The SOX demands firms to be accountable for their internal control procedures and their financial reporting system. Furthermore, the integrity of financial statements must now be analysed by an independent external auditor, who is responsible for ensuring that the financial information reported by the firm's management to stakeholders is free from material errors and is generated in accordance with the US. GAAP (GAO, 2006). In addition, the SOX lead to the creation of the Since companies are required to issue restatements to correct past reporting mistakes, restating activity may provide indications about the sources, the origins and the motivation of companies for providing poor quality financial reporting. Several authors have suggested that a high incidence of accounting accruals may be an indicator of low accounting quality. For example, Bradshaw, Richardson, and Sloan, (2001) and Sloan, (1996) suggest that high accrual level leads to an increase of information uncertainty, which causes an erroneous evaluation by investors since they are not able to use current earnings as an indicator of future earnings. In the same vein, Richardson, Tuna, and Wu (2003) suggests that companies are pressured by the markets to report positive results in order to attract external finance and/or lower interest rates. This pressure may lead them to enhance earnings management through the use of accruals. Therefore, the discretionary use of accruals by management might also be an indicator of the low quality of financial reporting. Dechow and Dichev (2002) developed a metric to evaluate accrual quality (AQ) and earnings quality. This paper submits that "observable firm characteristics can be used as instruments for accrual quality", and this metric allows to infer that "large accruals signify low quality of earnings, and less persistent earnings". A more recent study by Hribar, Kravet, and Wilson, (2014) suggests that the privileged access of auditors to clients accounting information, enables them to charge the auditing fee according to the quality of a client's financial information. It seems that the higher the amount charged, the lower is the quality of financial reporting. The authors find evidence that 'unexplained' audit fees are positively related to low accounting information and to a higher possibility of restatements and fraud.
Consequently, measuring financial reporting quality is likely to be related to several factors. Unlike the principles-based standards issued by International Financial Reporting Standards (IFRS), financial reporting in the United States is GAAP rule based. As a consequence, it is reasonable to assume that financial reporting in the US is not too much subject to professional judgement. Therefore, accounting quality will often be subordinated to interpretation and application of rules and principles, as well as incentives to manipulation, which also puts into question the ethical framework of those involved in the preparation and disclosure of financial statements.

Concept of Restatements
To keep its shareholders and the general public informed about its activities, publicly held companies based in the US have to regularly submit their financial statements to the SEC, which are then made available to the general public. This process is done by filling in multiple reports with different objectives. The Annual Reports on Form 10-K contains audited financial statements and a discussion of the results and performance of the company. To report unaudited financial statements for the quarters of the fiscal year Form 10-Q is used. These reports are used to make performance comparisons with the year counterpart. Finally, the filling of a Form 8-K happens when there is a need to announce events with significant impacts, such as the announcement of bankruptcy. The process of issuing a restatement may be triggered by the firm, by its independent auditor or due to an investigation headed by the SEC (Flanagan et al., 2008). When an error is found, the first order of business is to determine if it is materially relevant or not. Regarding this issue, Tan and Young (2015) note that "the FASB and the SEC provide several authoritative guidelines that discuss the establishment of materiality and the reporting of restatements of financial statements." When auditors conclude that previously issued financial statements contain material omissions or misstatements, the Generally Accepted Auditing Standards (GAAScreated by AICPA) require them to advise the client to make the appropriate disclosures, and to take the necessary steps to ensure this occurs (AICPA, 2002, Section AU 561). 2 Concerning the issue of materiality, ('material' or 'immaterial' omissions or misstatements) Tan and Young (2015) distinguish two classes of restatements: 'little r' and 'Big R'. The 'little r' is a restatement disclosed because of several immaterial errors that accumulate during a year until they become a 'material' error. "Big R" restatements, address a material error that calls for the re-issuing of a past financial statement.
Unlike a 'Big R' restatement, the 'little r' restatement "does not require an 8-K form or a withdrawal of the auditor opinion.". However, as noted by Chung and McCracken (2014), 'little r' restatements will have an impact on stakeholders since such an event will always raise doubts by investors and regulators about the quality of financial reporting.

Trends
According to the Government Accountability Office (GAO, 2002), during the period from 1997 to 2002, the number of restatements exhibit a steady increase resulting in losses of $100 billion on market capitalisation. A follow-up study by the GAO (GAO, 2006) shows that the number of public companies restating financial statements grew from 3.7 percent of the total listed firms in 2002 to 6.8 percent in 2005. As a consequence of this growing number of restatements and its market impact, the level of concern regarding the reliability of financial statements has never been higher (Hee, 2011).

In 2010, Audit Analytics (AA) issued the "2010 Financial Restatements -A Ten Year
Comparison" report, which looks at all the US financial restatements since 2001. Since then, reports are issued on an annual basis analysing how restatements evolve over time.
Currently, Audit Analytics holds a 'restatement' database that covers all filer types: 'accelerated' filers (restating companies that have at least $75 million in issued share capital, but less than $700 million); non-accelerated filers (firms with less than $75 million on public float); funds and trusts; new company registrations; small business filers and foreign registrants. Restatement records come from one of two sources: 8-Ks filed, or periodic reports. In 2013, Audit Analytics expanded its search process by reviewing the SEC comment letters from 2005.

'Stealth' Restatements
Files, Swanson, and Tse (2009) define three levels of 'prominence', i.e., the significance of a press release based on the GAO (2002) database and firms' press releases announcing the issuance of a restatement. The authors classify a restatement as 'highly' prominent if the press release is through a headline. Press releases referring to the restatement only in the body of the text are classified as 'medium', and those discussing the restatement in a footnote are labelled as 'low'. Files et al. (2009) shows that firms providing less prominent restatement press releases, which could be seen as 'stealthy', are less likely to exhibit a strong negative market reaction. This paper also shows that, in the post-event period, firms that have higher levels of analysts' coverage have their prices adjusted faster. In addition, Files et al. (2009) also finds that the probability of these companies being sued for securities fraud is low in the case of 'low' prominence press releases.
Hennes, Leone, and Miller (2008) uncovers relevant findings by relating 'stealth' and the importance of fraud in financial reporting. After analysing previous research on financial restatements, they conclude that these earlier studies often assume some stealth motives, i.e., that behind a restatement there is implied intentional misreporting (irregularities). However, these studies used restatements databases that include 'errors' and irregularities. Therefore, Hennes, Leone, and Miller (2008) contribute to increase the accuracy of research results by developing a method allowing the distinction of restatements between 'unintentional misapplications' of GAAP (errors) and intentional misreporting (irregularities).
On the 2 nd of July 2013, the SEC issued a press release 4 establishing The Financial Reporting and Audit Task Force. The main purpose of this task force is to investigate fraudulent or inadequate reporting, with particular emphasis on the Revision restatements (non 8-K restatements), which the SEC has considered more susceptible to fraud. Since one of the consequences of restatements is the loss of investors' confidence in financial reporting, it was considered that the issue of Revision Restatements instead of Reissuance Restatements, might improve confidence and also might mitigate some manager's 'nefarious' ways in their financial reporting. (Tan and Young, 2015) Using the analysis of Hennes et al. (2008), Kim, Baik, and Cho (2016) addresses the issues of sorting and detecting financial restatements. They developed three multi-class financial models to detect and classify misstatements according to fraud intention: multinomial logistic regression, support vector machine and Bayesian networks.
Further, they used cost-sensitive learning (using MetaCost) to improve detection and classifying.

Causes and Reasons
The analysis of the existing literature attempting to explain the content of restatements reveals that some researchers use the terms "cause" and "reason" as synonymous whereas other times they use these terms to explain what could originate and tend to the restatement. Section 2.2.3.1 outlines the causes -referring to the accounting issues -and the section 2.2.3.2 identifies the reasons behind restatements, i.e. what is in its origin.

Causes
The Sarbanes-Oxley Act of 2002 was enacted to mitigate investors' distrust towards corporate financial reporting. One of the consequences of SOX was the requirement for the implementation of various procedures in order to improve firms' internal controls concerning financial reporting (Weili and Sarah, 2005).
The 2006 GAO report analyses and describes the causes of restatements. Table 2.1 summarises the information provided by the GAO in that report:

Reasons
It is often assumed that a financial restatement is due to fraudulent behaviour. Yet, this may well not be the case. Plumlee and Yohn (2010) (2010) report that the remaining 37% are due to incorrect use of the accounting standards.
Related research shows that one of the reasons for weaknesses in internal company management controls for financial reporting is the lack of investment in qualified accounting workforce (Weili and Sarah, 2005). Similarly, Guo, Huang, Zhang, and Zhou (2016) finds that fair behaviour towards employees and motivation are the key to lessen the susceptibility of unintentional errors, and consequently reduce the event of a financial restatement. Jensen (2005) takes a complementary approach and analyses the "agency costs of overvalued equity" and presents the 'earnings management' as a major reason for the collapse of value in companies as well as the reason for the failure of others (e.g., Enron and WorldCom). In addition, Jensen (2005) argues that corporate managers are rewarded when they meet internal targets. However, the achievement of such internal goals is not the relevant variable to the markets as they reward or penalise the value of the firm depending on their performance in comparison to financial analysts' expectations.
In a subsequent study, Efendi, Srivastava, and Swanson, (2007) finds that the desire of Chief Executive Officers (CEOs) to hold in-the-money stock option is also an important issue in this context. In particular, the authors report that this increases the probability of the firm to disclose financial statements that are not aligned with the GAAP and, consequently, having to issue a restatement.
Board gender diversity may also shed some light on the reasons for restatements. In fact, Abbott, Parker, and Presley (2012)

Consequences
This section summarises some of the consequences of financial restatements, all of which are not directly related to the impact of such an event on financial markets: profitability, labour market and potential lawsuits.
Financial restatements are likely to be related to a decrease in firm's net income. This negative impact on profitability is not surprising given that about 40% of the restatements reviewed by GAO study of 2006 were due to deficient company revenue recognition. Plumlee and Yohn (2010) Lev, Ryan, and Wu (2008) show that restatements also increase the likelihood of lawsuits initiated by investors who realize that they have made decisions based on biased financial statements. As argued by Putman, Griffin, and Kilgore (2009), this is an important topic since, arguably, it calls into question the ethics of financial reporting as being a game, and consequently jeopardising the trust and integrity of the financial reporting system. Clearly, if restatements affect the reliability of financial reporting, company managers should strive to provide better forecasts after such an event so that they can regain their own, and the firm's reputation. According to Ettredge, Huang, and Zhang (2013), following a restatement, managers' behaviour is comprised of risk averting forecasting and consequently a loss of information on earnings content and a more conservative financial reporting is noted (K. Y. Chen, Elder, and Hung, 2014; Wilson, 2008).

Efficient Market Hypothesis
The impact of accounting events on the dynamics of financial markets has been a major issue in the accounting-based market research. In the context of this dissertation, it is crucial to understand whether the disclosure of a financial restatement information is efficiently assimilated by the markets. Efficient Market Hypothesis (EMH) advocates that financial markets assimilate immediately and without bias, any relevant information so that, in an efficient market, "on average, prices fully reflect all available information" (Fama, 1970). Fama (1970) discusses three categories of market efficiency: the strong, the semi-strong and the weak versions of the EMH. The 'weak' version assumes that prices represent all price-based 'historical' information. This weak-form contends that it is not possible to predict the future value of an asset, and, that any change in the price of that asset is due to 'unexpected' information. Consequently, it would not be possible for investors to obtain abnormal returns based on analysis of price-based historical information. The 'semi-strong' form implies that the market value of an asset adjusts immediately to all 'publicly' available information. Finally, the 'strong-form' of the EMH assumes that current price of assets incorporates all public and private information available. Under this very restrictive version, it is contended that investors cannot consistently generate abnormal returns even when they have access to privileged information.
The EMH has been severely criticised in the last decades by many authors who claim that it clearly fails to adhere to reality. Among the criticism, Behavioural finance (BF) has developed as a competing alternative theoretical framework, arguing that psychological biases and limits to arbitrage impede markets to work efficiently as prescribed by Fama (1970).
Consequently, restatements present an interesting opportunity to test to what extent the 'semi-strong' form of the EMH holds in real markets. Clearly, restatements provide crucial information that will potentially impact on the dynamics of financial markets and particularly on the current stock price of the announcing firm. In an efficient market, the impact of restatements should occur promptly and without bias, that is, with the stock prices of the announcing firms adjusting fully and quite rapidly following the disclosure of such public information. Conversely, any other scenario would indicate some failure of the EMH and would provide evidence supporting the arguments put forward by BF. Presley and Abbott (2013) study the overconfidence of CEO and the incidence of financial restatements. Among their findings, there is the evidence "that overconfidence is a relatively persistent phenomenon as it is a significant factor in the incidence of restatements in both the pre-SOX and post-SOX control environments."

The relevance of the study
This second chapter of this dissertation confirms that financial restatements constitute a relevant issue in the accounting and finance domain. This first approach to the restatements literature also allows concluding that financial restatements impacts on several important research topics such as accounting quality, regulators, investors, auditors, financial markets, etc.
The relationship between financial restatements and the dynamics of financial markets seems to be an important topic that can be systematically reviewed to structure existing knowledge on the importance of 'restating activity'. In addition, this systematic review allows the researcher to dive in the financial reporting area and the understanding of the most relevant topics in this area. More specifically, the 'importance of financial reporting' for capital markets and its relationship with the EMH and BF could provide future research avenues for subsequent empirical work as well as identify a potential contribution that can be developed on the author's PhD.
The primary goals of the systematic review are: 1. Design a research strategy allowing the identification of academic papers addressing financial restatements and its impact on the financial markets; 2. Identify and understand the most relevant issues in the connection between these two areas and the most recent developments; 3. Identify the gaps in the literature that offer research opportunities in future empirical work.

CHAPTER 3 -METHODOLOGY
This dissertation follows the systematic review methodology described in Tranfield, Denyer, and Smart, (2003). The systematic review process, as opposed to the traditional literature review method, employs an explicit and transparent method to identify, select and review the relevant studies related to the research topic.

Systematic Review Description
This section describes and explains the steps followed towards the accomplishment of a systematic review that minimizes the potential problems of a traditional review.
However it is important to notice that "there is no such thing as the perfect review" (Hart, 1998).

Theme
The identification of the research topic arises from the discussion with my supervisors.
In fact, we agree that the financial restatements area is a very relevant topic in the accounting and finance domain that could be systematically reviewed to identify potential research avenues to explore in a further stage (PhD). Although the existence of several research papers in this area, we believe that new challenges may arise from the discussion between financial restatements and efficient markets, behavioural issues, financial crisis or the need to increase investors' confidence in the dynamics of financial markets.

Scoping Study and Consultation panel
The scoping study is a crucial step to understand the main issues related to the theme to be explored in the systematic review (Tranfield et al., 2003). In addition, it helps to overcome some difficulties of an unexperienced researcher that is trying to give the first steps in this field. After some initial search on the electronic databases, we decided to explore the financial restatements only in the US market. This is because most of the relevant research on this topic is based in the US, because the US market is the most

Selection and Evaluation
• Inclusion Criteria • Exclusion Criteria

Analysis and synthesis
• Extracting • Selection

Findings
• Reporting of the Findings important market in the world and because companies operating in the US share the same legal environment. This choice ensures that the conclusions are robust and that the research opportunities are relevant to the academic community.
The creation of the Consultation group was an important step to overcome the main difficulties and the questions that were arising along the way. Table 3.1 identifies the members of this panel that are simultaneously the supervisors of this dissertation. The consultation group was essential to minimise the author's inexperience, to guide and supervise all the process of the systematic review, to find relevant conclusions allowing further empirical work and, most of all, to prevent some biases in the research.

Delimitation of research papers
This stage is divided into three steps explained in the two subsequent sub-sections.

Electronic databases
The search engine available at the University of the Algarve is B-ON, which aggregates several databases, such as EBSCO or Elsevier. The use of Social Science Research Network (SSRN) was also used since it is an important source of working papers on the fields of economics, finance and accounting.

Selection of keywords and search strings
The scope of the papers presented in Chapter 2 and the discussion with the consultation group is the basis for the author's selection of keywords to identify the relevant papers in this systematic review.  Keywords are next combined into six different search strings. These are listed below:

Search string 1: (financial AND restatement*)
This search string 5 is intentionally very broad and is designed to identify very general papers related to the main field of interest.
Search string 2: (financial AND restatement*) AND (fraudulent AND disclosure* OR restatement* AND announcement* OR accounting AND irregularities) This search string aims at identifying papers that specifically deal with restatements that are caused by irregularities and fraudulent practices.

Search string 3: (financial AND restatement*) AND ((financial AND market* AND (reaction OR impact))
This search string identifies papers that look at the impact of financial restatements on financial markets.

Search string 4: (financial AND restatement*) AND (shareholder* OR stockholder*)
This string find papers that look at the impact of financial restatements on shareholders.

Search string 5: (financial AND restatement*) AND (reputation)
This string attempts to identify papers that link financial restatements with the issue of a firm's reputation.

Search string 6: (financial AND restatement*) AND (share AND price*)
The last search string specifically searches for the impact of financial restatements on companies' share price.

Selection and Evaluation
A systematic review is a strategy that involves searches for keywords in electronic databases. The use of 'keyword searches' is a strategy of processing which has been used in several related academic papers. The first stage of this process is to identify relevant papers on the databases that are available for the purpose of this project. A second stage refines this first stage. It is based on the reading of titles and abstracts of papers so far identified and applies 'exclusion criteria' to reduce the number of papers to a relevant subset. This stage is followed by a complete reading of the final group of papers to ensure that all the papers in the final list match all the inclusion criteria defined in this systematic review. Finally, the presentation of results and the discussion of the findings is the final stage of the review.

Elimination of duplication
Considering that the search strings are applied to an engine browser that aggregates several databases, it is therefore important to remove the duplications. This process is made by exporting the results of each search string to the software Mendeley (v. 1.18), which automatically detects and eliminates repeated papers. The result is, therefore, a list of 'non-duplicate' academic papers.

Exclusion criteria based on the reading of titles and abstracts
The exclusion criteria summarised in Table 3.3 are then applied to the list of nonduplicate papers. This step aims at removing all contributions that lie outside the described scope and purposes of the systematic review. The criteria are applied to the 'title' and 'abstract' of each paper.

Table 3.3 Criteria and rationale for exclusion
Criteria Rationale

Articles published in other sources than scholarly journals
Financial restatements are referred to on a daily basis on the different media and other sources than scholarly journals. Since this is a systematic review of academic research, articles published in magazines and newspapers are excluded.
2. Studies that mention the defined keywords as residual issues or in other contexts than accounting and finance.
Some titles immediately suggest that a few of the identified papers are not relevant for the research. Hence, such studies were excluded at this stage.
3. Insufficient relation to be considered in the refined scope defined for the systematic review 3.1. Topics that are not directly related to the impact on financial markets Legislation that is not directly related to financial restatements and the impact on financial markets.

Topics related with financial restatements but approached from different perspectives.
Auditor litigation or auditor and management turnover, as well as corporate control events, are a consequence of Restatements but not directly relate to the impact on financial markets.

Studies based on markets other than the United States.
Main restatements database is from the US, and the study is only focusing on this market.

Inclusion criteria based on the reading of full text papers
Papers that passed the exclusion criteria are not automatically considered in the final sample. It is important to apply inclusion criteria to evaluate the quality of papers. This final step involves full text reading of the papers that pass the exclusion criteria, and that are tested against the theoretical and empirical criteria defined below.
Empirical papers must contain: 5. Discussion of the theoretical model's contribution.

Literature synthesis process
Consequently, as already indicated, the papers that pass all the criteria described above are included in the final sample and used in the systematic review. In this last step, the main findings of these papers are described and interpreted in the light of the following considerations: This chapter presents the methodological issues that are the basis of this systematic review and ensures transparency in the research process. The next chapter presents a descriptive analysis of the selected papers and reports the findings in the form of thematic analysis.

CHAPTER 4 -FINDINGS
This chapter presents and analysis the papers included in the systematic review process.
The analysis seeks to identify in each article the research hypotheses, the data used, the methodology and the findings. The chapter is divided into two main sections. First, it presents a descriptive analysis of the selected papers regarding the year of the articles and the respective SCImago 'Journals Ranking'. Second, the findings are presented thematically.

Process description
The processing of the search strings identified in the previous chapter provides a relatively large number of documents.  The total number of papers identified in these search strings are 651. The next step relates to the selection of papers described in the methodology ( As such, the number of academic papers without duplications is 394. The exclusion criteria (Table 3.3) were applied to this set of papers using successive 'subtractions'. As can be seen, the most important reason for exclusion is related to criterion 2, i.e., studies that use the keywords as residual issues or cover other areas than accounting and finance. The total number of papers resulting from the application of the 5 criteria defined in table 3.3. is 20. The final list of papers to review systematically is 19 since an additional paper was excluded based on the inclusion criteria defined in section 3.2.4.3  Appendix I presents a summary of those papers. That summary contains for each paper, its motivation, the methodology employed, the classification between empirical or nonempirical, the data used, the sample location and the main findings.

Report of the findings
The careful reading of the final 19 papers reveals that the impact of restatements on financial markets is significant and negative. The magnitude of the impact depends on several issues that are discussed in this section. Findings are presented in a thematic approach emphasising the most relevant issues in the connection between financial restatements and financial markets. The reading of these findings can be supplemented with the Appendix I, which provides for each of the 19 papers the motivation, the methodology employed, the classification between empirical or non-empirical, the data used, the sample location and the main findings.

Stock Market Reaction
The sections  obtain the CAR of a specific window. The period of an 'event window' related to the number of days around the event-date, which is defined as day zero. Therefore, a threeday event window around a restatement disclosure includes the prior day (-1), the day (0), and the subsequent day (1) to the announcement. This period is represented in square brackets [-1; 1] and the abnormal performance is verified when it is statistically significant.
Since these research papers analyse different time windows, investigate different reasons for the restatement and distinguishes from who starts the restatement, the report of the findings is divided into different topics.

Short-term
The papers that investigate the short-term impact of a financial restatement show an average negative impact in the days surrounding the disclosure. Moreover, there is some evidence of negative abnormal reaction before the disclosure date. This suggests that financial restatements represent a clear case of bad news to investors and that there is some anticipation of these news.    [-11,-2]. The evidence of negative abnormal reaction before the disclosure of the restatement raises suspicions about possible information leaks (e.g. Akhigbe & Madura, 2008;Hribar and Jenkins, 2004).   However,  underline that outliers do not influence their findings since a statistically significant percentage of enterprises in the sample display negative abnormal returns in the event-periods (60%).

Short-run market reaction according to the restatement cause
Using a different approach,   negative reaction is stronger in the case of restatements related to causes that are described as "easy-to-estimate" 8 items and which involve less estimation. On the contrary, the negative market reaction is weaker if the item restated is a "difficult-toestimate" 9 item. The author also takes into consideration the intentional and unintentional aspect of a restatement and interprets the fact of a company being sued as a proxy for fraud. Using an event window of 3 days centred around the restatement date, Salavei (2010) finds a more negative market reaction when there is litigation (without litigation) for easy-to-estimate items with a mean CAR of 13.02% (2.61%) and difficult-to estimate items with a mean CAR of 12.04% (2.88%).
In a parallel study,  finds evidence that restatements affecting multiple items and which review previously reported earnings are associated with more negative market reactions and that restating companies have reduced prospects.
Similarly,  finds that revenue and cost/expense issues are the most common causes for restatements in their sample firms (48% and 22% respectively) and that the 3-day negative abnormal reaction is 1.31% and 1.49% respectively.

Short-run market reaction: REITs vs non-REITs
In a more recent study,   and 2011 and compare the results with those for non-REITs. The authors claim that this is an important test as REIT's are more easily scrutinised and more transparent than non-REITs, and thus less exposed to information asymmetry and agency costs between managers and shareholders.  find a less negative market reaction to REITs' restatements (average negative CAR of 0.63%) than non-REITs (average negative CAR of 1.58%) over the [-1; 1] event window. Yet, further analysis shows that restating REITs with higher leverage and Book-to-market ratios experience a more negative market reaction 6.19% and 2.19%, respectively. Cox and Weirich (2002) highlights the impact of fraud not only in the profession of accountants but in society in general and point out that one of the reasons for fraud is 8 Revenue, cost and expense-related items. 9 Restructuring, securities related, Mergers and Acquisitions, and in-progress research and development.

Short-run market reaction: the issue of fraud
the pressure on managers and their attempt to beat the expectations of analysts. These authors examine a sample of 27 companies that committed fraud in the reporting of their financial statements between 1992 e 1999. This paper finds that shareholders of these firms lost 33 billion dollars during the event window around restatement [-1; 0] providing anecdotal evidence that firms involved in fraudulent reporting suffer a strong penalization in their value.
In a similar vein, Palmrose, Richardson, and Scholz (2004)  and consequently affect company's prospects. Chapter 2 of this dissertation identifies a paper that distinguishes between errors and irregularities (Hennes et al., 2008). These authors comment that the GAO database does not have sufficient data about fraud and the results found by , which used the GAO database sample, could be different if a different method was used to identify fraud. Nonetheless, while defining their method to identify fraud,  did underline that their fraud classification (which is based on firm's disclosure of fraud and enforcement actions by SEC) could produce biased result. First, because when companies find the need of a restatement, they issue a press release but may not mention that intentional misreporting is the reason, and second, because investigations carried by SEC could start pre or post announcement, and this timing (earlier or later) could bias the results.

Short-run market reaction: who initiates the restatement?
Palmrose et al. (2004) show that the abnormal returns in the three day window around the disclosure of a financial restatement depends on who initiates the restatement. The authors conjecture that restatements initiated externally can lead to more negative returns due to weak internal controls and management incompetence. Results show a negative abnormal reaction of 18% when the auditors trigger restatements, 13% when restatements are initiated by the company and only 4% when the SEC begins the process. In a related paper, Hribar and Jenkins (2004) also find negative and statistically significant abnormal returns for auditor-initiated (14.8%), and company-initiated (7.1%) restatements over the [-2; 2] windows. However, they find no significant abnormal reaction when the SEC-initiates the restatement.
More recently,  provide a possible explanation for the above scenario. During the year leading up to the disclosure, the CAR is negative regardless of who initiates the restatement. In the post-restatement period, the results indicate that the market response, in a three-day event window [-1; 1], is negative when the company (2.11%) or the auditor and the company (2.01%) prompt the restatement, but not different from zero if the SEC initiates the process. The justification given by these authors is that when SEC identifies an irregularity, the company rectifies the problem without fussing not letting the matter escalate.

Short-run market reaction: impact on risk
Previous studies show that the magnitude of the short-term negative abnormal performance is particularly pronounced when the restatement is initiated by the auditors or when there is a fraudulent behaviour . In the same train of thought, Hribar and Jenkins (2004) find evidence that analysts' forecasts one year ahead, are revised downward more sharply for event-firms with high growth in the past. Kravet and Shevlin (2010) justify the negative reaction in the stock price due to an increase in the risk component related to manager's discretionary actions (such as accruals) and also by enterprise characteristics (total assets, cash flow operations and sales).

Long-term market reaction to restatements
The long-term market impact of a financial restatement was also explored in some of the papers in the final list of this systematic review. For instance,  fail to find abnormal reaction in the one-year following the announcement.
However, this paper reports significant negative abnormal returns of 7.34%, 7.36%, 5.84% and 2,26% respectively in the four years following the disclosure.  also find negative abnormal performance in the year prior the announcement (9.6%). This result is explained by the possible poor performance of the company during the year before the disclosure, or to a potential market anticipation of the restatement. Concerning the period following the event, the values denote a growing market penalty by the year 4 (CAR of 21.86%).  also investigates whether the long-term impact of a financial restatement depends on the frequency that the company restates. The evidence suggests that, over the long run, the market reacts less negatively when a company is a repeat offender. Using cross-sectional regressions to, the long-term abnormal returns postannouncement show evidence that the market reaction is increasingly less negative the more frequently a company restates. The authors are unable to find a plausible explanation for this result and leave this as an open question for future research.
However, their logistic regression approach uncovers some factors that influence the odds that a company's restatement activity will re-occur compared to being a one-time offender. Factors such as size (an increase of 1% increases the odds to 17%), the listing Exchange of the company (in Nasdaq the odds are 51%), and a negative market reaction around the announcement of the restatement (odds of 44%).  shows that company's size explains both short-term and longterm reaction to a financial restatement. However, while the firm size shows that the bigger the firm, the more unfavourable is the market reaction in the long run, the short term indicates the inverse, suggesting that an initial under-reaction is corrected overtime. But, the same regressions "indicate that the long-term post-announcement market reaction is inversely related to the market reaction at the time of the announcement of restatements." This result does not allow one to conclude that there is consistency about any "over or under-estimation reaction by the market in response to financial restatement."

Cost of equity
Hribar and Jenkins, (2004) use analyst forecast revisions following an accounting restatement to account for the effect of the restatement on expected future cash flows.
Applying three different estimation methods, and depending on the model used, the authors estimate an average increase in the cost of capital that fluctuates between 7% e 19% during the month preceding the restatement. The same authors confirm that the capital upturns are more pronounced in the case of restatements which are auditor initiated (13.7%) than by the company (4.8%) or the SEC (1.8%). Firm's leverage level is also seen as a factor contributing to increases in the cost of equity (4.2%). One of the interpretations for these results is that investors are more concerned about a high level of debt and become alarmed when the auditor initiates the restatement, in the sense that it causes an increase of uncertainty and concern about the ability and the integrity of the company management. Bardos and Mishra (2014) augment the work of Hribar and Jenkins (2004)  Investors evaluate information risk according to the quality and quantity of information available that influence their decisions. Kravet and Shevlin (2010) studies the relationship between restatements and the cost of information risks. Specifically, they investigate if the 'discretionary risk' component associated with the decision-making of managers in accounting policies increases after the restatements, and the effect of this on the cost of capital. They argue that the 'evaluation' method for the cost of equity used by Hribar and Jenkins, (2004) may be biased on the part of analysts' forecasts. To conduct the study, they examine a time horizon of 6 years [-3; 3] using the quality of accruals and the use of accruals by managers to determine the 'information risk' and the 'discretionary information risk', respectively. The authors find evidence that the cost of information risk increases after the issuing of a restatement, and which results in an average increase of 0.86% to capital cost. However, they submit that the effect of an increase in cost of risk fades over the three years following the disclosure.

Cost of Debt
According to the Federal Reserve System between 1996 and 2006, the total bank financing reached a value of $780 billion, while the issue of equities has represented only $2 billion. Given these differences, it is important to understand how the cost and structure of private funding changes with restatements.  assess the impact of the restatements on bank financing as well as the effect on nonmonetary items. They compile a sample of 237 restatement firms with 2,541 loans, of which 1,568 started 'before' and 883 start 'after' restatements and use the period between 1989 and 2004. The results show that financial restatements impact on postevent banking agreement in terms of: Ø A higher spread: the penalty is higher for companies that issue restatements due to fraud with an increase of 68.9% (the increase for non-fraudulent restatements is 42.6%); Ø lower maturity: 17.1% (7.7 months); Ø increase in the probability of a loan insurance by 8.6%; Ø increase in covenant restrictions from 6.9 to 7.6; Ø decrease in the number of lenders: the number of lenders in a post-restatement loan decreases by an average of 6.5, compared with 8.5 before the restatement. This suggests that the perception of risk increases Kravet and Shevlin, 2010;, and that the resulting concentration of lenders allows better monitoring of borrowers. Park and Wu (2009) evaluate the cost of the debt using a different methodology. These authors argue that the results of  may include other factors that are not directly related with the restatements, due to the 'timeframe' between banking contracts. Through the estimation of abnormal loan returns, they find evidence that the loan market reacts negatively by increasing the spread. Moreover, this evidence is more pronounced when the restatement is started by the SEC or by the auditor.  do not find a statistically significant result regarding the 'prompters' of restatements. The results of Park and Wu (2009) confirm an increase in the bid-ask spread in the loan market on the three days around the event date (2.17% in day -1, 1.82% in the event day, and 1.87% in the day +1). A further analysis provides evidence that restatements related information arrives more quickly to the secondary market, which only later, is absorbed by the stock market. The authors justify this result due to banks' privileged access to the financial information of companies.
In a more recent study,  supports the results of both earlier studies, pointing out that the privileged access of banks to their client's information causes a higher spread (17.6%) and tight restrictions on loans, even before the issuance of the restatement. For loans 'after disclosure', the spread increases about 32.6%. The results lead the authors to conclude that banks' reactions to the information tend to be incomplete since there are further spread adjustments after the announcement of the restatement. The authors support this view because, although the spread increases, they do not find evidence of tighter non-pricing terms.

Restatements and firm growth
In the sequel of the results presented by , Kravet and Shevlin (2010), and Hribar and Jenkins (2004), a study conducted by , evaluates the impact of financial restatements in the company growth.
Using the method developed by Hennes et al. (2008) to distinguish restatements due to error or fraud, the authors analyse the relationship between companies that issue restatements and their internal and external growth. Where a restating company has its growth supported by external funding, these costs are adversely affected by 7.8%.
However, where fraud is the origin of the restatement the cost increase is 15.8%. These results incite shareholder value destruction to the extent that investment opportunities may be limited due to the increase in the cost of external funds.

Short selling
Short selling is a trading strategy used to profit from the expectation that the value of a stock will fall in the future. Thus, it is expected by 'academic' researchers that bad news events with significant negative impact in the value of company shares, such as restatements, may be related to short selling activity. This section reviews 3 papers that addresses issues related to short seller's behaviour in response to restatements: ; ; E.
Boyd, Marie Hibbert, and Pavlova (2014).  find evidence that short-sellers accumulate investment positions in restating firms long before an announcement by using a sample of restating firms and a control sample. For the event-firms, the average level of short selling eighteen months 'prior' to the event is 2.18%, 2.74% in the month following the event, and 2% eighteen months later. These results provide evidence that short-sellers unwind their positions in the post-announcement period, i.e., when the price declines. For the control sample firms, the paper presents stable figures around 1.6% in the period under review.

Short selling around restatements
Consistent with this evidence, a subsequent study performed by  reports relatively high levels of short selling in the month before the announcement of the restatement, when compared with the levels of short-selling of companies which are 'not involved' in any restatement activity.  justify the downward revision of earnings reported in earlier periods by the use of accruals.  contribute to this discussion by reporting a relationship between the high level of short selling and the low performance of these companies (often with a high rate of delisting). These results lead the authors to raise the hypothesis that short sellers are 'attentive' and capable of identifying questionable accounting practices of restating firms, i.e., suggesting that such investors are able to 'anticipate' the restatements.  also suggest that short sellers are particularly interested in companies issuing restatements to correct earnings previously reported and small companies that have weaker information environments, i.e., weaker financial management. High levels of short selling are more evident in companies that experience stronger negative returns in the 40 post-event days. High levels of short selling in the pre-event period is also reported and is explained by the sophistication of short selling traders that seem to be more vigilant and to follow closely companies where the quality of reporting accounts is weak. In this way, unlike , the authors argue that there is a 'reaction' rather than an 'anticipation' of restatements by short-sellers.

Short selling and Accruals
Given the legal requirements after the discovery of a mistake in a financial statement, the SEC has set a deadline of 4 days for the issuing of the restatement. During this period, the managers are forbidden to disclose information, suggesting that traders' anticipation of the financial restatements may be related to 'information leakage'. These short sellers can benefit from any initially incomplete or 'staggered' market reaction as described earlier.

Naked short sales and restatements
According to SEC, "in a 'naked' short sale, the seller does not borrow or arrange to borrow the securities in time to make delivery to the buyer within the standard threeday settlement period. As a result, the seller fails to deliver securities to the buyer when delivery is due; this is known as a 'failure to deliver'." E.  use the level of abnormal fails-to-deliver as a proxy for naked short selling and finds a significant increase in short selling activity both before and after the issue of a restatement. The increased short selling activity peaks on the 7 th and 6 th day before, and the two days following the disclosure. In line with the arguments of , the authors claim that the earlier first increase is related to the certainty that the shortsellers can anticipate restatements, and after disclosure the second peak moment is associated with the reaction of traders who may be less informed. However, this possible anticipation of restatements is documented only in respect of enterprises with high levels of institutional ownership.

Reputation
Karpoff, Lee, and Martin (2008)  The authors estimate that a company could lose up to 38% of its market value after the discovery of its financial misreporting. A more detailed analysis reveals that 24.5% of this loss is related to the adjustment of a new accounting reality, 8.8% is related to potential litigation from the SEC and shareholders, and the remaining 66,7% are due to the loss of reputation with their customers and suppliers.  find evidence that accounting restatements that have negative impacts on market value of event-firms also induce share price declines among nonrestating firms in the same industry (i.e. peer firms). The authors explain that this contagion effect is not related to revisions on analyst's forecasts but a lower financial reporting quality ("high accounting accruals and low operating cash-flows"). They also find an incremental penalty for the stock value of peer firms with similar accounting quality for those cases where the peer firms have the same external auditor as restating firms.

Financial restatements and industry effect
Akhigbe and Madura (2008) conduct a similar study to analyse the consequences on the market value of earnings restatements (ER) within a given industry. Examining a sample composed of restatement firms and a control sample of industry rivals, the authors find evidence that restatements which review previously recognised gains, cause a 'contagion effect' in the industry. This contagion effect within the industry is documented for earnings restatements that diminishes earnings previously reported as well to earnings restatements that reveal an improvement in previously reported earnings.
Cross-sectional analyses provide evidence that the adverse effects of ER are more prominent for highly concentrated industries and with a higher level of accruals.
Regarding contagion effect, restatements issued because of fraud do not produce a significant effect on non-restatement companies, since it is seen as firm-specific.
Another finding is that, since the Enron event, the industry responds more negatively to earnings restatements. In addition to these results, Kravet and Shevlin (2010) also provide evidence of an "intra-industry information transfer effect" in discretionary information risk, as an incremental reason for share decline among non-restating firms.
have on the assignment of a firm's credit rating, and in particular, the study evaluates the effect that the Enron episode had on the credit rating given to firms in the same industry (oil, gas and energy). The authors find evidence that severe restatementing (effect on net income, pervasiveness, number of years restated and the simultaneous announcement of news) relates to adjustments of credit ratings assigned by Standard & Poor's. Further, firms in the same industry sector as Enron which issued more harsh restatements were more penalized in their credit rating than restating firms in other sectors. Consequently, this result also justifies the presence of a contagion effect for peer companies in the ratings assigned by 'credit agencies'. The systematic review of the literature shows that the short-term impact of a financial restatement in the financial markets is one of the most explored questions. Several papers (e.g., Hribar and Jenkins, 2004;Akhigbe and Madura, 2008; find a significant negative market reaction in the days surrounding the disclosure of a financial restatement. In a more detailed level, the literature suggests that the magnitude of such negative reaction depends on several variables such as the cause or reason for the restatement, if the restatement is easy or difficult to estimate, if the restatement is with or without litigation, if the restatement firm already review previous reported earnings, if there was fraud or who initiates the restatement (e.g., Cox and Weirich, 2002;Kravet and Shevlin, 2010;. As such, it is important to explore other factors that may impact in this short-term reaction.

Implications for further research
Despite the consensus that financial restatements impact negatively on the short-term market value of event-firms, the evidence on the long-term market reaction is scarce and unclear (e.g., Gondhalekar et al., 2010). As such, this seems to represent an important research avenue to explore and clarify if the market fully and immediately assimilates the content of a financial restatement disclosure. Alternatively, the market may underreact to this bad news as in the case of other extreme accounting events such as going-concern opinions (e.g., Kausar, Taffler, and Tan, 2009).  highlights that there is no consistency about any "over or under-estimation reaction by the market in response to financial restatement". The behavioural finance approach with its limits to arbitrage and cognitive biases may have an important role in this research design.
Other important finding of this systematic review is that the market is able to, somehow, anticipate the financial restatement disclosure. In fact, some papers find significant negative abnormal returns in the pre-event period (e.g. Hribar and Jenkins, 2004; and that the short-selling activity also increases in the pre-event date (e.g., . Therefore, it is important to investigate the behaviour of other sophisticated agents that have the ability to impact in the value of firms like the financial analysts. One of the questions that can be explored in future empirical is whether financial analysts adjust their recommendations and their price targets in the months before the restatement announcement. In addition, analyst behaviour may contribute to augment the list of factors that impact on the short-term market reaction to the financial restatements. In particular, it seems important to test if the short-term market reaction depends on analyst opinion at the disclosure date.
The systematic review of the literature also uncovers that financial restatements have important consequences on the event-firm besides the loss in their market value. The literature shows that, following a financial restatement disclosure, the cost of capital increases, the cost of debt increases, the reputation of the company decreases and the firms operating in the same industry are negatively affected by this bad news event (e.g., Hribar and Jenkins, 2004;Akhigbe and Madura, 2008;Park and Wu, 2009;Bardos and Mishra, 2014;.
Therefore, it seems to be important to understand whether financial analysts react following the announcement date and if they continue to be interested in following such companies. This can be done by testing if the differences in their recommendations and price targets in the pre and post-event are significant and if the percentage of analysts that drop the coverage of those companies is statistically significant.

Limitations
In the author's opinion, the main limitations of the dissertation were the keywords choice and the definition of exclusion criteria. Although rationally supported and asserted by the consultation group, there is always an echo of the authors' interest areas and motivations in the definition of the exclusion criteria.

Methodology appraisal
The systematic review of the literature revealed as an important tool to avoid some weaknesses of the traditional literature review. The author became familiarised with the process, and in future uses of this methodology, although it is a continuous learning process, the increase in the learning curve of experience would be useful to reduce the 'time' spent during some stages.
Regarding the main purposes of this methodology, defined in chapter 3, it is indeed transparent and replicable by others. However, the process developed by the author is not free of criticism for several reasons. First, other researchers could define different keywords or argue that some are missing. Second, the reading of the title and abstract to select papers, may exclude some papers that other researches could include. Third, it can be argued that studies not published could enhance and give different perspectives to the results found.
Overall, the author believes that the methodology used, benefited the dissertation and reduced the level of possible criticism if compared to a traditional review process. Study's motivation: To determine whether earnings restatements prompt industry valuation effects.

Findings:
Ø Earnings restatements are associated with negative and significant valuation effects of rivals in the corresponding industry.
Ø Earnings restatements that have a favourable effect on the firm restating its earnings yield positive and significant valuation effects of rivals in the corresponding industry.

Findings:
Ø After restatements, the increase in the cost of equity is more pronounced and concentrated in sued firms.

Methodology: Multivariate regressions
Study's motivation: Contribute to a better understanding of the decision process of short sellers.

Findings:
Ø Short-sellers accumulate positions in restating firms several months in advance of the restatement.
Ø The increase in short interest is larger for firms with high levels of accruals prior to restatement.
Ø Short sellers pay attention to information being conveyed by accruals.

US Empirical
Methodology: Event study using Abnormal Failures to Deliver; and cross-sectional regression Study's motivation: Examine the relationship between naked short selling and accounting irregularities that cause a firm to issue a restatement.

Findings:
Ø Informed traders use the information flow from institutional investors following larger firms to anticipate the accounting restatements and serve as good market monitors of the firm.
Ø More transparent announcements are associated with more abnormal fails.

Findings:
Ø Evidence of a contagion effect resulting in a share price decline of non-restating firms.
Ø The contagion effect is more pronounced for peer-firms with high industryadjusted accruals and that use the same external auditor

Methodology: Regression analysis
Study's motivation: Study the effect of financial restatement on bank loan contracting.

Findings:
Ø Compared with loans initiated before restatement, loans initiated after restatement have significantly higher spreads, shorter maturities, higher likelihood of being secured, and more covenant restrictions.

Findings:
Ø The cost of equity capital average between 7% and 19% in the month immediately following a restatement.

Study 14:
Karpoff et al. Ø The increase on information risk, for restatement firms after a restatement announcement, results in an increase in the estimated cost of capital.
Ø There is an information transfer effect for non-restatement firms in the same industry.

Findings:
Ø Market reaction is less negative to restatements of difficult-to-estimate items