Week 75 — The End of Summer
Where did summer go? It’s already Labour Day weekend but this year being a COVID year is much different than other years. On the family front, my wife is returning to her job at a school under much different circumstances with some anxiety around whether schools will be safe or not. It’s too early to tell whether or not outbreaks at Ontario schools will happen or not but one thing is for certain those responsible for educating our children are not confident that the measures taken by the government so far are nearly enough. There seems to be a rather strange determination to keep class sizes at provincial mandated amounts which are far higher than the 15–20 that public health officials think are ideal to balance risk against the need to educate our children.
In other news, the Anti Racism Ambassadors Network is launching. I encourage everyone reading this blog to go check them out. The racism in Canada may not get as loud of coverage as racism in the United States but racism in Canada is real and impacts the lives of many BIPOC every single day. As an ally, you can amplify the voices of BIPOC people in your network or whose work you admire, buy goods/services at BIPOC owned businesses, call out racism when you see/hear it and educate other white people on the impact of racism and the actions they can take to be an effective ally.
Week notes have been less frequent as summer vacation and the start of school have taken over my life. I admit the writing “bug” hasn’t been as strong these days either. As a result, this update will be strictly about developments on my work with the hope that getting back into the habit sparks inspiration in the weeks to come.
Demonstration Projects
Rules as Code: In earlier weeknotes, I’ve mentioned how we are planning a full end to end (drafting, coding, release via public API) Rules as Code/Better Rules process. This past week we presented our project charter to all the partners on the project. Reaction was absolute enthusiasm and excitement to get the project started. While we anticipate we have more months until the drafting can start, there is work we are planning to do in the more immediate future such as concept modeling and decision trees for the existing regulation prior to doing the same with the new regulation (as it is being developed). This means that Canada’s full end to end Rules as Code project is officially kicked off! Note that unlike the earlier Rules as Code project I blogged about, this one will be slower since we are going through a drafting process. Nonetheless, I plan to share as much about the process as I can so all the readers of the weeknotes can follow along with the work.
In other Rules as Code news, we have three other Departments who have expressed interest in partnering on a Rules as Code project. Details are still being discussed but it seems that Rules as Code “has turned a corner” in Canada and awareness/support is rising. If you are reading this and want to reach out to talk about Rules as Code then tweet me @scott_pm or scott.mcnaughton@canada.ca
Machine Readable Regulatory Impact Analysis: We’ve completed our project charter for this exciting project (where we plan to see if we can convert the regulatory impact analysis into a machine readable format). In two weeks time, we will be hosting a user research/design workshop with regulators from 6 different departments so we can better understand how regulators would use a machine readable regulatory impact analysis to do better regulatory analysis. In the past, we’ve heard about the valuable data in these documents and how inaccessible the data is for advanced analysis (e.g. only available in HTML and PDF formats). There is a huge opportunity here to make this valuable data machine readable but only if we understand what kind of analysis regulators are doing and how a machine readable regulatory impact analysis would be used.
Essentially, we understand the what (solution) but we need to learn more about the why (problem) if we want to develop something that is useful for our target audience (regulators). If you are reading this and interested (e.g. a regulator) feel free to reach out because I want to talk to you!
Regulatory Evaluation Platform (and Peer Review of an AI System): We have started the final go home stretch for the Regulatory Evaluation Platform AI project. This means putting the polish on the system like translation, accessibility features, and other wrap up activities. At the same time though, we are exploring the concept of a peer review of an AI system. This is an interesting concept that can be found in the Directive on Automated Decision Making which states that every AI system that scores “risk level 2 or higher” must conduct a peer review. However, the details of what a peer review is, when should it be done, who should do it, how much should it cost, how long should it take etc. are unclear. So we teamed up with Treasury Board Secretariat and the University of Ottawa to do a mock peer review of the Regulatory Evaluation Platform. This includes a literature review, scan of best practices and the actual mock peer review itself. So far in the process, we’ve uncovered a lot of opportunities for guidance and best practices like advising Departments pursuing AI projects to budget peer reviews into their AI projects, anticipating the demands and requirements of a peer review and adding them into the project schedule and determining how “arm’s length” a peer review needs to be.
The literature review and the draft “peer review process” are completed. So now we are starting the mock peer review itself which is ongoing. When this project completes, I hope to share the results openly so others running AI projects can get an idea of what a “peer review” of an AI system looks like within the GOC.
Wrapping Up
So those are the weeknotes for Week 75. I wanted to use the space to talk more about the projects I am working on. In future weeknotes, I may dive deeper into other topics as I have done in the past.
Scott
