際際滷

際際滷Share a Scribd company logo

Sdmx at us federal reserve

Download as PPT, PDF
Vinicius Silva
Vinicius Silva

The Federal Reserve Board adopted the SDMX standard to improve how statistical data is organized and made available to users through their Data Download Program (DDP). While implementing SDMX presented challenges due to unfamiliarity with its data modeling approach, it provided benefits like an interface entirely driven by data structure definitions. Initial user feedback on the DDP has been positive, with over 250,000 visits since its launch. Future plans include adding more data sources and preparing for SDMX 2.0.

1 of 15
The FRB and SDMX: National data and International standards San Cannon Federal Reserve Board SDMXConference 9-11 January 2007
SDMX Conference 9-11 January 2007 2 Background The Fed is a statistical agency as well as a central bank and regulatory agency. Lots of data and information are available on the public website. Statistical data are varied: monthly industrial production indexes, daily interest and exchange rates, quarterly financial flows for various sectors of the economy, surveys of small businesses and consumers, etc.
SDMX Conference 9-11 January 2007 3 Serving our users better To some, it may appear that the statistical agency role is secondary. Data are not always easy to find. Downloads are not customizable. Example: Trying to extract one industrial production series requires retrieving two text files, cutting and pasting, reformatting. Complete yes. User Friendly no.
SDMX Conference 9-11 January 2007 4 Data Download Program (DDP) XML designated as key format but project team wrestled with implementation details. Staff weighed a homegrown DTD setup against the new SDMX standard. SDMX looked to have greater benefits and was adopted. Good decision: additional internal applications as well as interagency projects using SDMX are in the works.
SDMX Conference 9-11 January 2007 5 A lot to learn SDMX is based on data structure definitions (key families) and codelists, with every concept represented by a code with a corresponding definition. We were unfamiliar with this type of data modeling, so it proved challenging. Two of our pilot datasets translated easily to this new format; others needed more work.
SDMX Conference 9-11 January 2007 6 Data structures differ Some data structures are readily adapted to the concept/codelist representation. Series keys have no real mnemonic value. HBBA Int. Rate, Official, Discount rate/Base rate HBCA Int. Rate, Official, Intra-day loans SCBA Indust. Production, Motorvehicles, NSA SCBB Indust. Production, Motorvehicles, SA
SDMX Conference 9-11 January 2007 7 Hierarchical relationships We allow data to be modeled hierarchically, and use mnemonics that reflect this. RIFSPFF_N.B R.*:Rate R.I.*:Rate of interest in money and capital markets R.I.F.*:Federal Reserve System R.I.F.S.*:Short-term or money market R.I.F.S.P.*:Private securities R.I.F.S.P.FF.:Federal funds _N.:Not seasonally adjusted .B:Business (Five days, Monday- Friday) JQI_I02Y3361T3_N.M: J.*:Indices except of prices J.Q.*:Production J.Q.I.:Industrial _I.*:NAICS-based industry classification 02Y:codes from year 2002 3361.:Motor Vehicle Manufacturing T:thru 3363:Motor Vehicle Parts Manufacturing _N.:Not seasonally adjusted .M:Monthly
SDMX Conference 9-11 January 2007 8 Applying the SDMXmodel Data represented by a concrete number of concepts are much easier to represent with key family dimensions and attributes: JQI_I02YMF_N.M Topic_Industry_SA.Freq FA156900005.Q Prefix (2 digits), Sector(2), instrument type (5), series type (1), frequency Hierarchical relationships and varying number of concepts makes life more difficult: RIFSPPNA2P2D30_N.B Topic?_SA.Freq RIFLGFCY20_XII_N.B T opic?_Inflate_SA.Freq
SDMX Conference 9-11 January 2007 9 Decisions we made Allow a variable number of data structure definitions per dataset. Use the compact format for internal exchange and external downloads. Stick with SDMX 1.0, for now. Use a relational database to store data and XML information for retrieval.
SDMX Conference 9-11 January 2007 10 Final product: DDP! We have a flexible application whose interface is entirely driven by the data structure definitions. We store the XML as carefully sliced text in a relational database and we can build an index structure that allows us to respond to ad-hoc queries very efficiently, even for large volumes of data. Data Download Program
SDMX Conference 9-11 January 2007 11 Strengths and weaknesses Because interface is entirely data driven, it is easy to add new data. Internal architecture is complex, due to security and the data workflow: SDMX files are generated by data staff and transmitted to public website staff for processing. These files are made available on the website and shredded for database entry and lookup. Current structure is not set up for codelist sharing.
SDMX Conference 9-11 January 2007 12 What do the users say? Really excellent. One of the best Ive used on the web. This Data Download thing is better than sliced bread. I downloaded the XML files, but I cannot run them. All I see is the xml code.
SDMX Conference 9-11 January 2007 13 What do the numbers say? More than 250,000 unique visits since April about 50,000 per month. Data Download Program is the 6th most visited area on the Federal Reserve website.
SDMX Conference 9-11 January 2007 14 Next steps Add more data: reserves, exchange rates, consumer credit. Continue working with other central banks and statistical agencies on common framework. Prepare to move to SDMX 2.0 to take advantage of additional features.
SDMX Conference 9-11 January 2007 15 The last slide Questions? Comments? Thank you for your attention! San Cannon scannon@frb.gov (202) 452-3710

More Related Content

Sdmx at us federal reserve

  • 1. The FRB and SDMX: National data and International standards San Cannon Federal Reserve Board SDMXConference 9-11 January 2007
  • 2. SDMX Conference 9-11 January 2007 2 Background The Fed is a statistical agency as well as a central bank and regulatory agency. Lots of data and information are available on the public website. Statistical data are varied: monthly industrial production indexes, daily interest and exchange rates, quarterly financial flows for various sectors of the economy, surveys of small businesses and consumers, etc.
  • 3. SDMX Conference 9-11 January 2007 3 Serving our users better To some, it may appear that the statistical agency role is secondary. Data are not always easy to find. Downloads are not customizable. Example: Trying to extract one industrial production series requires retrieving two text files, cutting and pasting, reformatting. Complete yes. User Friendly no.
  • 4. SDMX Conference 9-11 January 2007 4 Data Download Program (DDP) XML designated as key format but project team wrestled with implementation details. Staff weighed a homegrown DTD setup against the new SDMX standard. SDMX looked to have greater benefits and was adopted. Good decision: additional internal applications as well as interagency projects using SDMX are in the works.
  • 5. SDMX Conference 9-11 January 2007 5 A lot to learn SDMX is based on data structure definitions (key families) and codelists, with every concept represented by a code with a corresponding definition. We were unfamiliar with this type of data modeling, so it proved challenging. Two of our pilot datasets translated easily to this new format; others needed more work.
  • 6. SDMX Conference 9-11 January 2007 6 Data structures differ Some data structures are readily adapted to the concept/codelist representation. Series keys have no real mnemonic value. HBBA Int. Rate, Official, Discount rate/Base rate HBCA Int. Rate, Official, Intra-day loans SCBA Indust. Production, Motorvehicles, NSA SCBB Indust. Production, Motorvehicles, SA
  • 7. SDMX Conference 9-11 January 2007 7 Hierarchical relationships We allow data to be modeled hierarchically, and use mnemonics that reflect this. RIFSPFF_N.B R.*:Rate R.I.*:Rate of interest in money and capital markets R.I.F.*:Federal Reserve System R.I.F.S.*:Short-term or money market R.I.F.S.P.*:Private securities R.I.F.S.P.FF.:Federal funds _N.:Not seasonally adjusted .B:Business (Five days, Monday- Friday) JQI_I02Y3361T3_N.M: J.*:Indices except of prices J.Q.*:Production J.Q.I.:Industrial _I.*:NAICS-based industry classification 02Y:codes from year 2002 3361.:Motor Vehicle Manufacturing T:thru 3363:Motor Vehicle Parts Manufacturing _N.:Not seasonally adjusted .M:Monthly
  • 8. SDMX Conference 9-11 January 2007 8 Applying the SDMXmodel Data represented by a concrete number of concepts are much easier to represent with key family dimensions and attributes: JQI_I02YMF_N.M Topic_Industry_SA.Freq FA156900005.Q Prefix (2 digits), Sector(2), instrument type (5), series type (1), frequency Hierarchical relationships and varying number of concepts makes life more difficult: RIFSPPNA2P2D30_N.B Topic?_SA.Freq RIFLGFCY20_XII_N.B T opic?_Inflate_SA.Freq
  • 9. SDMX Conference 9-11 January 2007 9 Decisions we made Allow a variable number of data structure definitions per dataset. Use the compact format for internal exchange and external downloads. Stick with SDMX 1.0, for now. Use a relational database to store data and XML information for retrieval.
  • 10. SDMX Conference 9-11 January 2007 10 Final product: DDP! We have a flexible application whose interface is entirely driven by the data structure definitions. We store the XML as carefully sliced text in a relational database and we can build an index structure that allows us to respond to ad-hoc queries very efficiently, even for large volumes of data. Data Download Program
  • 11. SDMX Conference 9-11 January 2007 11 Strengths and weaknesses Because interface is entirely data driven, it is easy to add new data. Internal architecture is complex, due to security and the data workflow: SDMX files are generated by data staff and transmitted to public website staff for processing. These files are made available on the website and shredded for database entry and lookup. Current structure is not set up for codelist sharing.
  • 12. SDMX Conference 9-11 January 2007 12 What do the users say? Really excellent. One of the best Ive used on the web. This Data Download thing is better than sliced bread. I downloaded the XML files, but I cannot run them. All I see is the xml code.
  • 13. SDMX Conference 9-11 January 2007 13 What do the numbers say? More than 250,000 unique visits since April about 50,000 per month. Data Download Program is the 6th most visited area on the Federal Reserve website.
  • 14. SDMX Conference 9-11 January 2007 14 Next steps Add more data: reserves, exchange rates, consumer credit. Continue working with other central banks and statistical agencies on common framework. Prepare to move to SDMX 2.0 to take advantage of additional features.
  • 15. SDMX Conference 9-11 January 2007 15 The last slide Questions? Comments? Thank you for your attention! San Cannon scannon@frb.gov (202) 452-3710

Editor's Notes

  1. Nice title.