�ݺ�ߣ

Editor's Notes

1. We face numerous decisions in life & business. APS similarly faces numerous decisions that carry with them uncertainty. The service needs for APS clients are often unusual and difficult to secure and fund. Both at the front line and administrative level, there is a need to be able to tell the clients’ stories so that potential donors and funders have an understanding of the complex issues involved in the protection of vulnerable adults. We can use computers to analyze the potential outcomes of decision alternatives. ECS turns uncertainty to a risk that carries a threat of loss due to abnormal costs in purchased items. One of the problems with carrying out sensitivity analysis is combining the individual sensitivities in the model, of which there may be many, in a way that realistically reflects the range of possible outcomes that may occur in practice. A simple approach that is often used is to set all the sensitivities to the worst case, and look at the results that this gives. However, by doing this the downside of the project is vastly over-stated; in reality, it is unlikely that everything will go wrong together. Even if realistic combinations of sensitivities are used, simply applying these in the model does not provide any information about how likely, or probable, the resultant outcome is. WHY MONTE CARLO? The technique of Monte Carlo analysis overcomes these problems: it allows the user to combine many sensitivities in many different ways, effectively running hundreds, or even thousands, of scenarios. The output shows the range of possible results, and how probable each is. This information enables the user to understand the effect that variations in the inputs have on the output: in other words, this method makes clear the risks involved in the project.
2. I have two primary goals in this presentation: One is to look at the cost/per/client probability distribution The second is to link data tables during the CARE and ECS data process
3. The output of my simulation gives the cost per client at the service category level and allows one to check for anomalies when comparing actual fiscal year expenditures per client to simulation results through the use of QA trigger thresholds - Risk audits may be triggered when thresholds are exceeded.   I also include a sensitivity analysis chart that designates which service codes (at a state level) impact the variance of the calculation the most – i.e., magnitude and direction of that correlation.    So for example, if after running the simulation for APS Assistance (service category) and identifying a threshold amount over which APS management may not feel comfortable with, one could look at the more detailed sensitivity analysis to figure out which service code would be contributing the most to that result and review that service code level to reduce the threat of loss.    
4. Bubble Diagram 1 displays the overall structure of the model. The Excel files to go with your packet have a copy of the models and data. I will start from the left and move towards the center. One thing I want to point out that is important is the definition of probability distributions Defining Probability Distributions for Input Variables Historical Data Probabilities with small samples Short term focus I used expenditures by service code to come up with my input distributions for total cost for each service category, e.g., APS Assistance. Expert Feedback Knowledge of spending and procurement patterns in APS
5. Now that we have discussed the model itself, I want to highlight some settings I used for the calculation. SOFTWARE: Crystal Ball software is a leading spreadsheet-based software suite for modeling and Monte Carlo simulation. (Oracle Website, 2009) One of the key capabilities that CB provides is the automated generation of random inputs from within the established input distributions. It is these random numbers that represent specific values of the given variable during each iteration of the simulation. # OF TRIALS USED: The precision of the output value and the distribution shape improve as the number of simulation iterations increases. Note: To ensure appropriate accuracy in Contribution To Variance view, consider running at least 10,000 trials. WHY PROBABILIYT DISTRIBUTIONS: By assuming probability distributions, I will be defining not just the range of likely outcomes but the likelihood of each arising;
6. Crystal Ball compares the number of values in the certainty range with the number of values in the entire range to calculate the certainty level for the forecast. By default, Crystal Ball calculates the certainty level based on the entire range of forecast values. The certainty level is one of Crystal Ball’s key statistics because it shows the probability of achieving the values within a specific range. The selected lower limit here was $382.75 – none of the historical fiscal years reached this level of expenditures. Actual Average Cost Per Client FY05 - 345.17712 FY06 - 348.35359 FY07 - 354.40574 FY08 - 333.26085
7. CORRELATION COEFFICIENT Crystal Ball calculates sensitivity by computing rank correlation coefficients between every assumption and every forecast while the simulation is running. Correlation coefficients provide a meaningful measure of the degree to which assumptions and forecasts change together. If an assumption and a forecast have a high correlation coefficient, it means that the assumption has a significant impact on the forecast (both through its uncertainty and its model sensitivity). Positive coefficients indicate that an increase in the assumption is associated with an increase in the forecast. Negative coefficients imply the opposite situation. The larger the absolute value of the correlation coefficient, the stronger the relationship. To help interpret the rank correlations, Crystal Ball provides a default chart view called the Contribution To Variance view. This view makes it easier to answer questions such as "What percentage of the variance or uncertainty in the target forecast is due to assumption X?" It is important to note that the Contribution To Variance method is only an approximation and is not precisely a variance decomposition. Crystal Ball calculates Contribution To Variance by squaring the rank correlation coefficients and normalizing them to 100%. Both the alternate Rank Correlation View and the Contribution To Variance view display the direction of each assumption’s relationship to the target forecast. Assumptions with a positive relationship have bars on the right side of the zero line. Assumptions with a negative relationship have bars on the left size of the zero line. APS ASSISTANCE Here you can tell that the # of clients variable has the largest contribution to the outcome, with a negative correlation. So as this amount increases, the amount of the forecast decreases. From the service codes, 36R and 36A are the largest contributors.
8. Actual Average Cost Per Client – two were outside the trigger threshold I came up with…costs were in the 25 th percentile for the state. FY05 - 356.3439512 FY06 - 355.7954837 FY07 - 405.4050617 FY08 - 394.4135923
9. When we look at the contribution to variance for this category, the 36M:Medication service code is significantly contributing to variance. That may be due to the large cost variance when the Medicare B program came about and clients went down…making costs in the calculation go up.
10. Actual Average Cost Per Client – none of the fiscal year measurements for the previous four fiscal years were outside the trigger threshold I set up. FY05 - 1226.320111 FY06 - 1274.744359 FY07 - 1133.031604 FY08 - 1072.964187
11. Here again, the # of clients variable is the largest contributor to variance in the calculation. It is followed by 36K, H; 37I, 35R.
12. Actual Average Cost Per Client – two of the fiscal year measurements for the previous four fiscal years were outside the trigger threshold I set up. FY07 and FY08 cost per client were higher than 75% of other costs. FY05 - 733.4556306 FY06 - 669.4732505 FY07 - 889.1346435 FY08 - 893.3435579
13. When we look here, we can tell that the 36E: Housing Repair/Modification service code has very high variance in comparison to other costs. Looking at the FY08 costs, APS spent over a million dollars for this service in the state. It may merit further exploration to bring this cost down by better negotiations on price.
14. Actual Average Cost Per Client – three of the fiscal year measurements for the previous four fiscal years were outside the trigger threshold I set up. FY06, FY07 and FY08 cost per client were higher than 75% of other costs. FY05 - 73.07 FY06 - 478.40 FY07 - 409.24 FY08 - 321.19
15. Looking at the sensitivity chart, we can see that 35U contributes a lot to the variance in the results. Further examination may be warranted and will be discussed in segment two of my presentation.
16. Actual Average Cost Per Client – only one of the fiscal year measurements for the previous four fiscal years was outside the trigger threshold I set up. FY06 cost per client was higher than 75% of other costs. FY05 - 284.43 FY06 - 557.8125 FY07 - 73.8 FY08 - 0
17. The 35L expenditures contribute the most to the cost per client outcome. It could be explored further to reduce the variance and the costs. # of Clients Historical Client Volume Data FY05 4 FY06 4 FY07 3 FY08 0 Service Codes 35L Descriptions Legal Services Historical Expenditure Data FY05 1,137.73 FY06 2,231.25 FY07 221.40 FY08 0.00
18. Actual Average Cost Per Client – two of the fiscal year measurements for the previous four fiscal years were outside the trigger threshold I set up. FY05 and FY06 cost per client were higher than 75% of other costs. FY05 - 42.94 FY06 - 37.59 FY07 - 31.81 FY08 - 18.34
19. The 36I expenditures contribute the most to the cost per client outcome. It could be explored further to reduce the variance and the costs. Data # of Clients Historical Client Volume Data FY05 4,219 FY06 4,095 FY07 3,953 FY08 5,360 Service Codes 36I Descriptions Administrative Fee (Claims Processing) Historical Expenditure Data FY05 181,173.91 FY06 153,948.36 FY07 125,744.31 FY08 98,310.30
20. The structure of this model is similar to the previous mentioned with minor changes – this gets at a service code level analysis with regional values as input variables to the distribution.
21. Actual Average Cost Per Client – none of the fiscal year measurements for the previous four fiscal years was outside the trigger threshold I set up. FY05 - 815.19 FY06 - 774.36 FY07 - 781.94 FY08 - 790.63
22. The two highest factors contributing to variance for this service code are the number of clients variable and the Region 6 distributions. More emphasis can be placed on those two measures to increase comfort with the parameters use to capture those input variables to reduce variance.
23. Actual Average Cost Per Client – two of the fiscal year measurements for the previous four fiscal years were outside the trigger threshold I set up. FY05 - 223.25 FY06 - 216.90 FY07 - 198.13 FY08 - 176.05
24. The two highest factors contributing to variance for this service code are the number of clients variable and the Region 6 distributions, once again. More emphasis can be placed on those two measures to increase comfort with the parameters use to capture those input variables to reduce variance.
25. Actual Average Cost Per Client – one of the fiscal year measurements for the previous four fiscal years was outside the trigger threshold I set up. FY05 - 1657.55 FY06 - 1875.26 FY07 - 1386.40 FY08 - 1289.37
26. The two highest factors contributing to variance for this service code are the number of clients variable and the Region 8 distributions. More emphasis can be placed on those two measures to increase comfort with the parameters use to capture those input variables to reduce variance. Region 8 36K Residential Care 44,499.88 98,653.15 89,378.03 63,770.00
27. Actual Average Cost Per Client – one of the fiscal year measurements for the previous four fiscal years was outside the trigger threshold I set up. FY05 - 672.41 FY06 - 724.84 FY07 - 898.12 FY08 - 960.08
28. The two highest factors contributing to variance for this service code are the number of clients variable and the Region 1 distributions. More emphasis can be placed on those two measures to increase comfort with the parameters use to capture those input variables to reduce variance. Data # of Clients Historical Client Volume Data FY05 862 FY06 681 FY07 795 FY08 1156
29. Actual Average Cost Per Client – one of the fiscal year measurements for the previous four fiscal years was outside the trigger threshold I set up. FY05 - 672.41 FY06 - 724.84 FY07 - 898.12 FY08 - 960.08
30. The two highest factors contributing to variance for this service code are the number of clients variable and the Region 10 distributions. More emphasis can be placed on those two measures to increase comfort with the parameters use to capture those input variables to reduce variance. Region 10 35U Emergency Response Svc 0.00 3, 862.83 587.50 420.00
31. Actual Average Cost Per Client – one of the fiscal year measurements for the previous four fiscal years was outside the trigger threshold I set up. FY05 - 227.55 FY06 - 371.85 FY07 – 73.80 FY08 - N/A
32. The two highest factors contributing to variance for this service code are the number of clients variable and the Region 10 distributions, once again. More emphasis can be placed on those two measures to increase comfort with the parameters used to capture those input variables to reduce variance. It may be Region 10 35L Legal Services 359.50 1,342.00 125.50 0.00
33. http://www.dfps.state.tx.us/about/state_plan/2005-2009_Plan/06.asp All service delivery and case planning is supported by the IMPACT system. Standardized templates are used for preparation of child and family service plans. Resource Management The IMPACT system also serves as the repository for information regarding foster and adoptive homes, service providers, and available resources. The system tracks foster care and adoptive home applications and licensing decisions in addition to any abuse/neglect involving foster or adoptive homes. The IMPACT resource directory contains information regarding available resources and services statewide. Information about service providers also supports financial management and contract management. An automated interface with the Health and Human Services Administrative System (HHSAS) supports payments to a wide variety of service providers. Caseworkers complete service authorizations for clients, which are then used as the basis for billing/invoicing of service providers. Unique Functions of IMPACT Aside from supporting statewide access of case information, the IMPACT system is used for a variety of data tracking and monitoring purposes. The IMPACT application has 156 web pages that support data entry of approximately 6000 data elements stored in more than 300 different tables in the database. IMPACT supports the maintenance and monitoring of an array of programs, including the Data Warehouse, the Court Improvement Project, and tele-working. 2) Section two of my presentation involves a discussion of what is needed to link the different data element tables coming from IMPACT. Specifically, how to capture the “problem” [CARE/Outcome Matrix] to be solved by the ECS service code captured in a service authorization. ·         I propose that it is an important first step in linking expenditures to client data, but more importantly, a means to capture the decision trends at a regional and state level for how APS specialists are using the APS service codes available to solve particular problems . ·         In a way, this is the foundation for an APS knowledge management system that is readily documented and available for management to make informed decisions regarding quality assurance goals and staff use of available tools.   ·         This would provide quantifiable data that could facilitate a broader consideration of more closely grouping the CARE items to the service code list used in the service authorization. Without this first step, the transition is far more difficult and the change more drastic.
34. This is a high level process map that captures the progress from intake to closure, specifically focused on the ECS action selecting and resulting service authorization to expend APS funds. Part of the motivation for my project has been to strengthen resource development efforts to better meet the needs of our clients. To do so, it is helpful to track how the current in-house ECS funds are being used to solve particular client problems. A place to begin is by exploring what service codes are linked to the solving of root problems and to what extent. The current data capturing system in IMPACT does not facilitate this linkage and is a weakness for the APS data system.
35. http://www.asp.net/LEARN/sql-videos/video-104.aspx A different way to look at the previous process map is this data flow graphic. The data tables between the Outcome matrix and Service Authorization are not linked in terms of causal problems – the APS specialist makes a decision as to how to apply a solution, but there is no readily obtainable documentation of the corresponding CARE problem.
36. http://www.asp.net/learn/sql-videos/video-106.aspx One promising solution to the gap is the use of relational databases. Relational database tables – linking them through use of keys. (Can be accomplished with Visual Web Developer 2005 Edition) primary key vs. foreign key Primary key – allows you to differentiate a single row of data (e.g., license plate); person id serves that purpose in IMPACT. Foreign key – allows us to designate one or more records in the second table with one record (primary key) on the first table. Key fields – relating the Outcome matrix table to the service authorization table. identity column applied to the primary key column in the data tables; case id# is likely a key that can tell cases apart Person ID/SA ID = primary keys; Problem ID/Person ID/Case ID= foreign keys (relate back to Person ID # in previous OM table) Service Code information would be kept under SA ID# and related back to Outcome Matrix. The relationship linking maintains data integrity between the two tables – no orphaned values. Problem ID field – captured from the Outcome matrix tables automatically.
37. Linkage then becomes possible.
38. This change allows for the consideration of a stronger APS knowledge management system to analyze resulting ECS decisions. knowledge management embodies the strategies and processes that a firm employs to identify, capture and leverage the knowledge contained within its “corporate memory”. knowledge management is a business process . It is the process through which firms create and use their institutional or collective knowledge .”
39. This is going on all over the state. Some points regarding this decision map are presented.
40. http://www.mindtools.com/dectree.html Decision Trees are useful tools for helping you to choose between several courses of action. They provide a highly effective structure within which you can explore options, and investigate the possible outcomes of choosing those options. They also help you to form a balanced picture of the risks and rewards associated with each possible course of action. This makes them particularly useful for choosing between different strategies, projects or investment opportunities, particularly when your resources are limited. Decision trees provide an effective method of decision making because they: Clearly lay out the problem so that all options can be challenged. Allow us to analyze the possible consequences of a decision fully. Provide a framework to quantify the values of outcomes and the probabilities of achieving them. Help us to make the best decisions on the basis of existing information and best guesses. A strong institutionalized decision analysis process in APS would add support to strategic initiatives in APS to align the program to meet tomorrow’s rising needs. It is an important component to support new resource development initiatives and networks. Example based on FY 08 APS data: Expenditures # and % of problems by action category Clients served
41. MCS allows APS to analyze the effect of distributions of input assumptions, rather than just fixed values or ranges • understand the combined effect of a number of different risks A case can be made to the State oversight authority to ask for additional funds for particular services. • Having statistics, a specific budget and documentation of need will enhance credibility.