Observations from the September 26th JLARC I-900 Subcommittee public hearing for the SAO's performance audit of the WSLCB.
Here are some observations from the September 26th Washington State Joint Legislative Audit and Review Committee (JLARC) I-900 Subcommittee public hearing.
My top 3 takeaways:
- The Office of the Washington State Auditor (SAO) presented their performance audit of the WSLCB titled Improving Cannabis Risk Management Tools Using Business Transaction Data (transcript, audio, video, slidedeck).
- Deborah Stephens, SAO Senior Performance Auditor, described the two questions the SAO performance audit sought to address: “First of all, what data could help LCB identify high-risk business transactions or business transactions where the, the risk of, sort of, illegal activity was particularly high? …And then, how could they use [traceability] tracking data that they collect to focus their audit and enforcement efforts on those high-risk transactions?”
- Stephens outlined the boundaries of the performance audit, a work examining processes and data from the BioTrack era: “The audit did not examine the reliability of LCB’s new tracking system [MJ Freeway Leaf] or the quality and completeness of its data. Nor did the audit examine LCB’s effectiveness at implementing the new system. …The audit did not examine the extent to which diversion of cannabis occurs among licensees. Finally, the audit did not examine the effectiveness of LCB audit and enforcement efforts beyond their use of available data.”
- The SAO modeled the Washington cannabis industry supply chain, identified transaction points where risks of illegal activity were deemed high, and assessed how to use existing data collection to alert regulators of unusual activity that may merit further investigation.
- High Resolution Process Maps (8.5mb PDF)
- The SAO defined high-risk transactions as “points in the process—the production, processing, or retailing process—where the risk of removing cannabis or cannabis products from the regulated market to the black market is particularly high.”
- When cannabis changes form (e.g. flower/trim to an oil extract)
- When cannabis or products are transferred to another licensee, sold to a customer, or gifted to a patient
- When licensees adjust quantities or weights
- SAO Performance Auditor Jenna Noll explained that BioTrackTHC’s seed-to-sale tracking software did not identify unusual data, and the agency largely relied on complaints to help target investigations.
- The SAO performance audit recommends WSLCB define ranges of reasonable data values for each high-risk transaction point. Then, WSLCB could require MJ Freeway to code or activate existing features to generate alerts when transactions stumble outside the normative range of expected values.
- Desired outcomes for WSLCB are to increase regulatory compliance while optimizing usage of staff resources and increasing excise tax revenue.
- The WSLCB was given an opportunity to present and respond to questions from the I-900 Subcommittee members (transcript, audio, video).
- Pat Kohler, Deputy Director and executive sponsor for the traceability project, explained how WSLCB switched traceability vendors to MJ Freeway during the course of the audit which began in 2016. WSLCB assured the JLARC members that the new system will be more flexible, with better analytical and notification capabilities.
- Kohler acknowledged missing features and defects will continue to be addressed the rest of the year. She added that some requested changes are core to the system’s design (e.g., changing global IDs) and will not be addressed within the scope of the current contract.
- Kohler reiterated the agency’s formal response to the audit: the WSLCB intends to implement the SAO’s recommendations by the end of the year.
- Representative Christine Kilduff asked if the WSLCB would need to be granted greater authority to conduct audits or investigations. Kendra Hodgson, WSLCB Cannabis Examiner Manager, said she did not believe so and the agency would comply with relevant laws and rules in the development of its data irregularity alert system.
- Representative Ed Orcutt, committee Secretary and Acting Chair, asked if the agency is using their current, incomplete system to investigate violations. Kohler said WSLCB Enforcement is currently using Leaf traceability data as one source of information, and that information is confirmed or refuted by licensee records and field observations.
- Representative Orcutt asked if the agency would be open to having the SAO audit the new traceability system. Kohler said the agency was open to having the SAO “…come in and review the system.” So readers may witness a performance audit of MJ Freeway Leaf in 2019.
- Members of the public offered testimony prompting questions from Subcommittee members (transcript, video).
- Steve Sarich of the Cannabis Action Coalition testified to the lack of compliant medical products and MJ Freeway’s track record. He argued a recent bust by WSLCB showed they had lost the handle on traceability compliance and suggested the agency conduct more random sampling of producers to “scare the bejesus out of these people” and convey “there’s a new sheriff in town” (transcript, audio, video).
- Gregory Foster, testifying on behalf of Cannabis Data and The Cannabis Alliance, raised questions about the recommended algorithms, offered recommendations for implementation, and reiterated industry concerns about the integrity of the state’s traceability data (transcript, audio, video, written testimony).
- He noted the SAO’s recommended algorithms for identifying high-risk transactions made some questionable assumptions:
- Lab-measured potency values are assumed to be reliable, and input material potency values are assumed to be comparable to output material potency values though they may be calculated by different labs.
- Potency is assumed to be consistent throughout an input lot.
- Unit weights are assumed to be identical.
- Foster offered three recommendations to the WSLCB:
- Require flexibility in implementation to enable WSLCB staff to more quickly change threshold values to avoid false positives
- Create and publish a Standard Operating Procedure (SOP) for investigation of high-risk transaction alerts
- Gather and publish anonymized statistics on outcomes of alert-driven investigations
- Jim MacRae from Straight Line Analytics testified about the end of publicly available traceability data post-BioTrack and its impact on policy and planning. He also discussed gaps in lab quality assurance which could hinder assessment of high-risk transactions (transcript, audio, video).
- MacRae blasted WSLCB’s and MJ Freeway’s inability to deliver the Leaf product on time: “…when they launched on February 1st, there were some pretty serious problems that were supposed to be fixed in a hotfix by February 28th – one month. Some of those fixes—and not all of them—were finally executed on August 28th. But, but it’s still not up to the original specs of the contract. It takes 12 to 18 months to do a project like this. Well, it’s been 14 months now, and we’re not still there. I am hopeful that by the end of the year they will be there, but I do believe, from a risk mitigation perspective, we should recognize that there is at least a possibility that this system is gonna fall flat on its face, and it’s simply not going to be able to feed the type of proactive assessment of the market that the auditors’ report has suggested.”
- MacRae noted a quantitative approach could also be applied “…[to] proactively see if labs are going out of line with respect to their reporting of potency, to see if labs are going out of line by not failing any product for, for quality assurance reasons.”
- Orcutt pursued a line of questioning about potency testing by the Washington State Department of Agriculture (WSDA). Under an Interagency Agreement between the WSLCB and WSDA for pesticide testing of cannabis samples, WSDA tests an estimated average of 75 samples a month. Orcutt asked if MacRae had acquired data on discrepancies between lab and WSDA reported results. MacRae explained WSDA sometimes includes cannabinoid potency results in their reports, but samples are anonymized. He added that WSLCB must possess information associating WSDA test results with licensees and possibly specific lot numbers.
- Sarich suggested that constraints on WSDA’s oversight authority increases use of harmful or banned pesticides on cannabis products, further disincentivizing patient participation in the commercial market.
- Foster observed, “[Steps] in the direction towards making this more of a sustainable and thriving industry are good ones that the industry would welcome. And I want to acknowledge the fact that the LCB has recently opened up rulemaking on quality assurance of products and doing another round of adjustments to the regulations around pesticides and encouraging, as well, seeing if they can make changes to address structural deficiencies in the marketplace that have impacted the prevalence and the availability of medical product.”
Follow Up (Nov 6, 2018)
Here are documents from the September 26th JLARC I-900 Subcommittee public hearing obtained via public record requests.
- September 26, 2018 JLARC Meeting – LCB Follow-up. At the end of the hearing, the acting Committee Chair, Representative Ed Orcutt, asked WSLCB to provide written follow-up to the JLARC members and staff regarding comments and questions raised during the Improving Cannabis Risk Management Tools Using Business Transaction Data report presented by the State Auditor’s Office (SAO). Deputy Director Pat Kohler sent this official agency response on October 30th, 2018.