Week 27 — Data Analytics for the Public Sector
Presenting on the Regulatory AI Projects and Talking About Measuring Outcomes And Not Outputs
For week 27, I found myself on travel status as I visit Toronto for 3 days to attend and present at the Data Analytics for the Public Sector Summit. I was invited to participate in a panel to discuss measuring public secotr innovation and to present on the Regulatory AI projects. Before I get into the formal weeknotes I wanted to do a small section on the government travel system, perhaps the most poorly designed internal government system ever imagined. For those of you who have been unlucky enough to use “HRG” the government travel system, you know the pain that the travel system causes. The system is horribly designed hiding basic functions under non-logical menu options using language that is not familiar to anyone who is not a travel expert. It uses a lot of symbols which appear to have no relevance to the function they perform. The urban myth is that the travel system was designed to be intentionally obtuse so people would call for telephone support (which comes with a much higher service fee than booking online). I have no idea whether that is true but you have to imagine we can do better. So I wish the NextGen travel system team all the best as they attempt to fix a system that is wasting countless hours of public servant time as we navigate and use a system not designed to allow for efficient booking of travel.
Data Analytics for the Public Sector
Over the past two days, I had the pleasure of attending as well as presenting at the Data Analytics for the Public Sector summit. It’s a two day conference focused on artificial intelligence, data analytics and business intelligence. The conference organizers did a good job of recruiting different kinds of speakers who covered everything from case studies, interesting presentations, fascinating panels and more to create a dynamic and thought provoking event.
With that being said, I wanted to share some of the key points I made during my presentation and panel at the event:
- Diversity is important if we want to be better public servants designing services that are effective and help people. However, diversity in our workforce is only the first step. Public servants are not representative of the people they serve. Public servants are largely speaking university educated, making way above the average income and live in urban environments (or close to them). This is not representative of the average Canadian.
- It is important for public servants to get out of their bubble and talk to real Canadians. If you don’t, you create an echo chamber that will not let you do your best work or design the best services.
- Failure is ok and it is good to keep re-enforcing this message. However, the structural systems (budgets, at-risk pay, who we promote etc.) of the public service do not indicate a tolerance for failure.
- Bias in AI is bias in our people, processes and systems. We need leaders who understand that the problem is not that the AI is bias but that we are bias. We need leaders who can face that head-on and be open to hard truths.
- Innovation succeeds because you have people who understand how to take something risky or novel and navigate it through the bureaucracy. The talent gap in government isn’t in hard skills (we can hire AI developers). The talent gap (if it exists) is in people who understand how to move through the bureaucracy while understanding innovation. It’s a rare skillset and we need to build up people to be able to do this.
I also presented on the Regulatory AI projects. The deck can be found here. The core message throughout the presentation was to showcase how it is possible to bring AI into regulations. And if the regulators can start experimenting with AI then what is your excuse?
AI Demonstrator Projects (Incorporation by Reference and Regulatory Evaluation Platform)
Incorporation by Reference: A close-out meeting for Phase I was held last week. While some minor refinements are needed, we are ready to move to Phase II of this project. As mentioned last week, we plan to release the source code on Github with an open source license so others can build on this work and fork the project to their hearts content.
Regulatory Evaluation Platform: Work continues with the platform as the contractors continue to work out how our requirements emerge in real life. A surprising revelation during this project was the realization that the unpacking of the “Regulatory Stock Review” exercise was more complicated than we expected to be. In many cases, we are being asked questions that are difficult to answer because many regulators have never done the exercise so it becomes a theoretical exercise (e.g. how would you do it today if asked to do it) rather than focusing on how it is done today (because in many cases it isn’t being done right now).
Rebuilding the Public Service From The Ground Up: Week 14
Week 14 is dedicated to a topic of conversation while I was at the Data Analytics Summit. Conversation has focused on measuring outcomes vs. outputs.
Idea 13: Better Performance Measurement
The public service is not the best at defining key performance indicators and measuring the success of our programs. Often, we are focused on outputs (e.g. how many widgets did we make) rather than focusing on outcomes. Many in government are also struggling to develop indicators which can demonstrate impact.
So this idea is not radical but we need to train public servants on how to be better at measuring impact and focusing on outcomes. We need to change incentive structures to put more priority on making sure our work is doing what it is supposed to and specifically to ask whether outcomes were reached. Far too often, we ask if we made enough widgets when the real question to answer is whether those widgets were used to help save lives. So if we change the incentives and train people, then we can shift government to be impact and outcome focused rather than output focused. It would mean changing training and performance agreements. It would mean unlearning a lot of what we have taught ourselves but it is possible.
How about as a starting point, everyone develops one outcome based indicator for the current project they are working on? I’ll start: for the REP project, we need to make regulatory modernization more efficient so why not say that the REP project needs to cut down the amount of time to do regulatory modernization by 90%?
Those are the weeknotes for week 27. I hope you have a good week and see you soon!