Semantic Analysis of Canadian Regulations

This project combines expertise in legislative drafting and legal data science to conduct an automated semantic analysis of more than 2700 Canadian federal regulations. It investigates four legal characteristics of regulations: (1) Prescriptivity, (2) Flexibility, (3) Complexity, and (4) Age. Drawing on the literature in relation to the assessment of regulatory burden, automated analysis of legal text, and plain language drafting, we devise a conceptualization and operationalization of these four characteristics. In order to scale the legal analysis of these characteristics, we rely primarily on an easily extendable and transparent dictionary approach that searches through the corpus of regulations for the presence of signaling terms and phrases. Our main insights and findings include:

Prescriptivity: How binding are regulations?

• We define prescriptivity as a relative measure of prescriptiveness (“shall”, “must”, ect.) in comparison to permissiveness (“may”, “entitled to”, ect.). Canadian regulations are becoming more prescriptive. Although prescriptive terms themselves are decreasing, this is offset by a larger drop in permissive terms which render regulations more rigid.
• Prescriptivity tends to vary by regulatory subject matter. Consumer product regulations tend to be very prescriptive (see e.g. Carriages and Strollers Regulations scoring highest on our count) whereas other regulatory fields such as procedural court regulations tend to display low prescriptivity. Additional grouping will be necessary to determine whether, within a group of similar regulations, a document is overly prescriptive. When this is the case, it often signifies either poor drafting (e.g. unnecessary repetition of prescriptions) or burdensome regulation.

Flexibility: How responsive are regulations to changing circumstances?

• We investigate three aspects of flexibility: (1) exceptions (on regulatees), (2) discretion (for regulators) and (3) incorporation by reference. We find that these dimensions have remained constant over time, but that they vary strongly across regulations.
• Similarly to precriptiveness, flexibility varies systematically across the type of regulations. Whereas financial or transport regulations, for instance, have more opportunity to incorporate external industry standards, other types of regulation contain few such references.

Complexity: How easy is it to understand regulations?

• Complexity, in our definition, captures how accessible the regulatory text is to its readers. This is achieved through plain, clear and well-structured drafting. Unfortunately, off-the-shelves readability measures (e.g. Flesch Kincaid) do not accurately capture the accessibility of regulatory texts since regulations are formatted and structured differently than the texts these readability measures were developed for.
• Using a word list approach instead, we find that regulations become more complex in their structure through more internal cross-references, but are less reliant on legal jargon.

Age: What is the average age of regulations?

• We calculate how much time has passed since a regulation was last amended, but also look for references to outdated technology. We find that some regulations specify what means of communication are permitted. This excludes some newer forms of communciation such as email in older regulations that have not been amended recently.

There remains work to be done to continue developping a better understanding of regulations. This work includes validating and refining our measures, resolving outstanding issues like a regulations’ readability, and using our measures to create best practice frameworks within a regulatory environment.