This section sets out what can be done while undertaking the analysis to ensure that your user needs will be met and can be communicated appropriately in your report.
A trustworthy and useful report requires the underlying work to be carried out with sound processes and controls that keep the user in mind throughout. Earlier planning and user analysis will ensure that you are not under time pressure and will be able to carry the user on your shoulder as you do the work.
Keeping the report in mind during the analysis will help ensure the right work gets done and in a way that is useful for later reporting. Sometimes following a process mean going back a step before moving forwards. Be prepared that something will be discovered during work that requires additional planning or engagement with users during the analysis. Good notetaking and preliminary exhibit creation will make later reporting a snap. These ideas are discussed in greater detail below.
If you keep the report in mind while performing each stage of the analysis, then you are more likely to complete them to the degree needed by your report. For example, if you know your report will require sensitivities, you should conduct your analysis in a way that allows you to capture a good range of sensitivities. Doing this at the time of the analysis will help you select the most meaningful scenarios. Also, this effort will help you later if your writing uncovers further scenarios are needed.
The following table sets out example topics by analysis stage which are potentially of interest to the report. It is not suggested that all of this detail be included in your report. Instead, this level of detail will help you decide as the work develops what will be important to complete to a reportable standard. If pre-planning was done to a very detailed level, it is only the execution that matters, but that may not have been practicable, or things may have been evolved. The first time you try it, you may only adjust for a few issues, but through writing, you may uncover ones you want to handle better next time.
|
Modelling Step |
Topics that might be reported |
|
1. Problem and approach statement |
What question is being answered? What is the problem(s) the work will address? What are the past and future trends/shifts in the problem space? What ideas/logic are underlying the approach? What is excluded or is of particular uncertainty? |
|
2. Systems/Model Changes |
The specific approach and platform used and any changes since the prior update. Back testing vs prior model Any simplifications used. Key risks & uncertainties. |
|
3. Assumptions setting |
Specific and relative assumptions on trends, credits, etc., tests of assumptions, risks & uncertainties. |
|
4. Data |
Sources, changes, adjustments, data rejected, uncertainties |
|
5. Model set-up and running (including parameter selection) |
limitations, checks, segmentations, diagnostics and selection logic (why those ldf’s or weights), what changed, issues log |
|
6. Output/consolidation of results |
final selections, allocations/summary to Stakeholder relevant groups. Transformations to different accounting bases. Useful tables. |
|
7. Analysis and checking of results |
Trends and relationships in output. Outlier or shift checks. Reconciliations, comparison to management info and forecasts. Sensitivity testing, analysis of change, alternative model results, Peer review, External benchmarks. story of why selections make sense. Standards met |
|
8. Reporting |
Reporting tables (expected in a regular format or proscribed, charts that help with an explanation. risks & uncertainties. subsequent updates, change analysis, uncertainties, contradictions, comparative benchmarks. |
|
9. Project Management |
Process and controls followed, timelines, notes/ issues log |
As you are carrying out the work, all material surprises and challenges are best communicated to your users at the time they occur.
This helps build trust, since early communication enables the users to understand the implications of the results and give you feedback or additional information or resources to help you. How they talk about the issues will also help you know what language, reasoning and details will be satisfactory in later reporting. If these interim communications are documented sufficiently, then this may simplify the information required in the final report as a conversation can be referred to rather than fully explained. Next steps that were not possible to carry out and their reasoning may be reported as well.
The discussions may also let you practice your summary of key issues, facts, and conclusions in plain language, and make their drafting in the report easier.
If you are not able to communicate with your users directly, consider how they would react to the results you have produced and what they would value from the actuarial report.
For all work, documentation along the way saves time and effort in report writing. While documenting, include everything but also think about why it was done and how the work and any results will be received by your users.
The documentation should allow you to remember where things came from, what issues were encountered, decisions made, uncertainties remaining, and actions recommended.
"Good documentation is not just what, not just how, but also why… (it) is a more comprehensive picture of the model, the assumptions, and the thought processes behind the choices made." (1)
The following tips are taken from a useful article by Slope software on Good Actuarial Model Documentation Practices
This will give you standards and practices you can employ, refine, and hold professionals accountable to. Don’t be afraid to let this document evolve over time.
Include what you’re supposed to do after a meeting to the 45 minutes you spent debugging that one add-in, make some notes when you’re finished; it doesn’t have to be super-detailed. You won’t have to divert time and mental energy away from your current projects to think about the past.
Go back through all your in-the-moment documentation, refresh your memory, and create a final, “pretty” version for your peers and supervisors. This will give you the opportunity to pull together all the disparate nuggets of wisdom scattered throughout however many pages you’ve been making notes on through the development and validation processes. Doing it right away will avoid that risk of knowledge erosion.
https://slopesoftware.com/2019/12/26/good-actuarial-model-documentation-practices/ accessed on 04-Sep-2021
Read the other blogs in this series on Good Actuarial Reports.
