Skrinja (Chest): Using Emerging Technologies for Better Digital Public Services and Data Driven Decision Making in Slovenia
The innovation has been developed to support data-driven decisions and to improve transparency and efficiency for better governance. Manual analytic work can be better organised and automated by offering self-serving dashboards and infographics. Data Warehouse and Business Intelligence are used as a central governmental platform to be offered as a service to the public sector. The project is introducing new concepts and tools of analytics and business intelligence in the public administration for better services.
In 2017, the pilot project “Big Data Analysis for HR efficiency improvement” led to the establishment of a central data warehouse and business intelligence called “Chest”, which had the aim of supporting data-driven decision making to improve transparency and efficiency for better governance. The BI “Chest” concept follows GDPR requirements and Information Commissioner directions regarding personal data security. The first data source which is being imported to the Business Intelligence system is the Slovenian public salary system (called ISPAP) which comprises the whole salary system in the Slovenian public sector. The second data source which is being prepared to be imported to the Business Intelligence system is the system of public procurement data.
According to the team's experiences, some of the key aspects of Business Intelligence implementation are change management and support of top management. Business Intelligence brings new possibilities, different ways of thinking and eliminates some of the old patterns in the working process. Since proper data interpretation is of utmost importance, the collaboration of expert users & data owners is very crucial for the success of the Business Intelligence implementation. Therefore IT experts work closely with content experts, speak the same language, share the same ideas and follow common goals when implementing a Business Intelligence system.
Business Intelligence as a digital platform enables multiple collecting, linking and analysing different databases, seeking different patterns, finding new knowledge and information from various aspects in real-time reporting processes. It provides new data dimensions, patterns, combinations and possibilities to create new information and their visualisations which isolated databases cannot provide. Moreover, it saves time and money by supplementing different repetitive reporting using Excel. The majority of this manual work could be predefined and automated by offering self-serving dashboards and infographics. Consequently, analysts could rather focus on smarter tasks e.g. on predictive analytics, ad-hoc reports, "what if" future scenarios and proper data interpretation to offer a good basis for better planning. On the other hand, Business Intelligence provides a solid system operating as “one truth”, meaning that reporting is done automatically using original data sources, which prevents errors in reporting.
The Business Intelligence system as a central digital governmental platform is intended to be offered as a service to other governmental agencies. It will enable solid, quality information for governmental users aiming to support better decisions both on an operational, tactical and strategic level. Data owners will decide to what extent their Business Intelligence results will be provided to public, but all such inputs will increase transparency and provide additional data for reuse. The team intends to also design a competence centre offering professional support and knowledge dissemination to public administration users. The focus has been given to data security and protection according to GDPR together with national personal data legislation. Therefore, the data warehouse is comprised of separated data marts for each source, each accessible by required authorization and authentication process. Data marts can only store pseudonymised personal data and process of pseudonymisation must be done before loading data to data mart (privacy by design).
Data warehouses and Business Intelligence are not a novelty in business and industry, but to date, they have not been used extensively in governments. Especially in the form of a general platform and as a horizontal service. This is a different environment and many issues like data structure requirements, safety, privacy, and authorizations had to be specifically addressed to provide trustworthy procedures and technology on many different government data sources.
The Chest is providing a powerful tool for government institutions enabling them to better and faster analyse their data. Results can be used in decision making and provide a trustworthy and transparent source given to the public and business for reuse. By simplifying technical hurdles and shortening development procedures, more innovative reports and graphical representations can be prepared, thus giving previously hardly accessible information to the public and business. For example:
- Transparent information about salaries in the public sector can dissolve many doubts about government efficiency.
- Public procurement data can be used by business to better prepare their offers for public tenders and providing a good source for market analysis.
We can expect that integrating other data sources on the Chest platform will provide even more useful information, not just for government officials but for the public and industry too.
What Makes Your Project Innovative?
The private government cloud platform is used to address concerns about safety, security, and privacy in Slovenian public administration.
The Chest is providing shortcuts by taking care of user needs, hardware, network, system software, tools and providing good practices. These simplifications encourage institutions to engage in activities which until now were too expensive, time-consuming and require special technical skills.
The Chest is not designed to accept any personal data whatsoever. Data source owners should pseudonymise them according to their needs in their distribution environments, conforming to GDPR requirements.
Government officials can access only data provided by their institution according to legal basis and GDPR. However, some certain public published reference data, code lists and classifications (apart from personal data) can be used by all users. Such data is uploaded, managed and organised as Common Dimensions by The Chest team.
What is the current status of your innovation?
Two data sources (Salaries and Public Procurements in the Public sector) with common dimensions have been implemented at the moment.
The first source is being tested and is scheduled for production in March 2020. The second source is in the development phase. Business requirements were documented and data uploaded.
Many lessons have been learned, good as well as bad practices experienced:
- top managers must strongly support organising data in Chest. Presentations and use cases have been organised;
- involvement of personnel with expert knowledge about their data is crucial. Their task is to select proper data, prepare data description, organize their data in the distribution environment, execute data masking and perform data testing including desired reports and dashboards.
- data source owner must grant authorizations on different levels to users.
- data source owner must verify and decide which results can be publicly accessed.
Collaborations & Partnerships
The partnership of public administration, academia, and private companies. MPA officials designed conceptual solutions and business specifications. University of Ljubljana (UL), Economics Faculty helped us with expertise in gathering business requirements and made the first model plan and draft concept of the system.
Contractors: Qubix, Result, Gora and B2 provided skills and knowledge on emerging technologies like hardware and software requirements and data science and engineering skills.
Users, Stakeholders & Beneficiaries
Public managers will have online reliable information to questions regarding public salaries and procurements, giving better foundations for their decisions. Selected reports will be publicly available, giving citizens and civil societies transparent information.
Public procurements Business Intelligence shall be accessible to companies giving them the better starting point for tenders offers and better information about market analysis. New data sources will expand data domain analysis and answer more questions.
Results, Outcomes & Impacts
Business Intelligence system solution as a platform and competence centre for user support is to be organized for public institutions. Competence centre staff will share experiences, knowledge, project governance and management, provide guidelines on security, care for personal data and trust-building. Besides IT users – non-IT people from other government institutions need to contribute their knowledge and competence about the content of new data sources in the future.
Due to GDPR and legal requirements, data are organised in domain, data marts, and common dimensions. Special care is being given for personal data protection according to GDPR.
Users can focus on data analysis instead of data gathering and preparing.
With the first two data sources, data impacting ¾ of the national budget is already covered. The project has not yet been finished, but there is already substantial interest in government institutions to increase the number of data sources.
Challenges and Failures
- Recognition that government institutions need a different approach to business intelligence.
- Data must be organized in separate data marts and common dimensions because of data ownership demand by GDPR.
- Use authorizations are granted by data owners.
- Data exchange between data marts is possible but must be agreed on and well documented.
- The Chest is such a project that will provide many tangible results at the end of the project. Confidence and patience by sponsors are essential.
- Project management must be different since the focus is on data not coding.
- Hardware, software and license types must be properly defined.
- A lot of data and methodology inconsistencies have to be addressed in collaboration with data owners experts. The team leaned on knowledge and expertise from business and academia (Institute Jozef Stefan, Faculties for Computing and Economics).
- Project is new in public administration therefore some challenges had to be overcome, delaying project schedule.
Conditions for Success
Since the team is establishing a new type of project in public administrations, draft rules and needed tasks are being recorded to be used as guidelines for adding new sources. Cooperation, knowledge, and expertise from business and academia were crucial for the beginning phases of the project supporting an emerging technology. Because of the multi departmental and multidisciplinary nature of the project, efficient project management is very important. In this respect it is also important for users & content experts and technical experts to closely work together. The nature and value of data in domain data marts demand secure, personal data protected and trustable environment for business intelligence, otherwise public institutions would not allow their data to be analysed in such manner. With additional data sources, face a lack of staff issues might arise, since data professionals are hard to attract in public administration. The project is co-financed by the EU and is valued at 235.000 €.
The concept of the project is to build a platform for public institutions, which will save them expenses due to shared data sources, infrastructure, and good practices.
Two data sources are to be delivered at the beginning of 2020. Many other institutions have already expressed their interest to join Chest. This confirms that our intent to provide a service for government institutions as the competence center was correct. The purpose of business intelligence competence center is open and neutral regarding content to be analyzed.
Government institutions function differently than enterprises, therefore data need to be organized and managed differently. The team organized and established separate data marts and common dimensions and provided process and methodology for data owners to grant authorizations to users with required attention for personal data. These conclusions are the result of many visits to relevant companies and gathering their experience and consulting academia institutions. The team engaged professors for sharing good practices and for help in preparing initial documentation.
A lot of time was spent in the initial phases of the project to gather experience and build documentation. Now it is considered a good and crucial investment to avoid dangerous pitfalls and errors in the future.
It is to be expected that each new data source will bring with it some data inconsistencies which take substantial effort to be addressed.
An interesting lesson learned concerns official or base registry & information system for salaries and number of employees in public sector – which surprisingly posed more obstacles than anticipated. Through transformations of data to compatible form for analytical model several issues arose. Inconsistencies (in data quality, data consistency and data access) were shown for some code lists and other reference data, for certain measures/facts there weren’t uniform methodologies for calculating or reporting them, there were some issues with historical data especially regarding code lists and reference data. Additional problems in preparing model and later analysis were detected because some organisations did not report: the necessary data, corrections and several comments or metadata remarks. There is a strong need for closer and more active cooperation among public sector bodies and for more precise and common guidelines regarding data governance (owner/controller of data, data management) for master as well as reference data.