Cognoa Success Story

Acting faster on childhood behavioral conditions

As many as 1 in 10 children have a mental, behavioral, or learning issue that can negatively impact their education and create a barrier for integration in the wider community. Despite the high prevalence of these behavioral challenges, many families find it a struggle to access timely support. Pediatric behavioral health company Cognoa is on a mission to change that by enabling faster, more equitable access to care.

Jeff Miller, Senior Director, Head of Engineering at Cognoa, says: “Across the USA and around the world, many families don’t have access to behavioral specialists, resulting in missed opportunities to identify students with specialist needs. Early diagnosis is really important, because interventions before the age of five can significantly improve long-term outcomes.”

Introducing an AI-powered solution

Cognoa has pioneered an innovative diagnostic aid for autism spectrum disorder (ASD): Canvas Dx. The solution is designed for children aged 18-72 months at risk of developmental delay, and combines data from three sources: a questionnaire completed by the child’s caregivers, a questionnaire completed by a healthcare professional, and two videos of the child interacting in their natural environment. Using AI, Canvas Dx then evaluates the data and delivers the results to the child’s physician to support the clinical decision-making process.

“Initially, we developed our solution for use in non-clinical settings,” continues Miller. “As we trained the AI and machine learning models that underpin Canvas Dx, its accuracy grew. Our senior leadership team recognized that the solution had the potential to transform the way that physicians diagnose ASD, and decided to pursue approval from the Food and Drug Administration [FDA].”

Pivoting from start-up to medtech

To turn its publicly available app into a prescription-only software-as-a-medical-device (SxMD) product, Cognoa had to ensure full compliance with a wide range of requirements. In particular, the company needed to show the connections between its requirements, specifications and designs, and the test cases associated with each.

Miller explains: “We had to ensure that our systems and processes were 100% compliant with FDA requirements. As the first step, we looked to implement a design control system that would allow us to ensure full traceability throughout the development process.”

Adopting an agile approach

After considering solutions from several leading vendors, Cognoa selected MatrixALM—a simple and efficient requirements management solution that helps organizations capture specifications, tests, risks and more in an agile way.

“Some members of our team had worked with the Matrix Requirements solution in previous roles, and their feedback on MatrixALM was very positive,” comments Miller. “Unlike many other application lifecycle management solutions, MatrixALM offers great flexibility when it comes to integration. Using native APIs, we can use data from the Matrix Requirements solutions in other platforms, and bring data from external platforms back into MatrixALM in a streamlined and automated way.”

He adds: “Most importantly, the solution allows us to trace a single requirement all the way down to the individual test results that link to it—helping us to demonstrate to our regulator that we are working by the book.”

Automating thousands of tests

Today, Cognoa has deeply embedded the Matrix Requirements solution into its development process. In preparation for each new release, 11 employees including engineering teams and product and quality specialists leverage MatrixALM to help test and validate software components against FDA standards.

“We utilize Ruby on Rails, Swift, and other open-source software to develop Canvas Dx, and our end users run the solution on a wide range of iOS and Android platforms,” explains Miller. “To ensure regulatory compliance and deliver an excellent experience for end users, it is crucial to thoroughly test new software releases.”

In total, Cognoa completes approximately 5,000 test cases as part of each new release cycle. In addition to running hundreds of tests in-house using MatrixALM, Cognoa partners with an external data quality engineering services company, which carries out and stores test results in its own TestRail platform.

“In other solutions for SxMD development, the handling of test cases feels like an afterthought—but not in MatrixALM,” says Miller. “Using a combination of scripts and Matrix Requirements APIs, we can import our partner’s test outputs directly into MatrixALM—maintaining full traceability between our test cases and their associated requirements. Without automation, there would be no practical way for us to ingest this data, and we’d be missing a valuable opportunity to demonstrate our commitment to quality to the FDA.”

Driving continuous improvement

Working with experts from Matrix Requirements, Cognoa has further enhanced its approach to third-party integrations. By configuring the Agile Sync module in MatrixALM based on its specific workflow requirements, the company can automatically detect, review and correct issues such as data duplication or mismatches before importing test data back into the solution.

“Matrix Requirements was super responsive when we asked for help importing our test workflows,” says Miller. “Whenever we have a questions or technical issue, the team always goes out of their way to provide a fast resolution.”

Bringing an innovative solution to market

With fine-grained traceability MatrixALM driving its requirements management process, Cognoa has achieved market authorization for Canvas Dx—and the solution is now helping physicians across the country to diagnose and treat ASD faster.

“Because we are building a SxMD product, it’s vital to show the FDA that we take activities such as testing seriously—and that we have solid data behind us,” elaborates Miller. “Without question, MatrixALM has helped us to shift our attitude to development: we’re now very test-driven. Our commitment to quality doesn’t come at the expense of agility. With workflow automation from MatrixALM, we can import thousands of test cases from our third-party provider in just hours— 95% faster than if we were performing the same work by hand.”

Staying agile

An agile business, Cognoa delivers new releases of its software solution every eight weeks. As the company enhances the solution with new capabilities, MatrixALM makes it easy to identify and apply the required changes to documentation.

Miller comments: “Recently, we added support for two-factor authentication via SMS to the Canvas Dx app. With MatrixALM, it was easy to look back through our specifications to find the design objects affected by the change, and discuss the potential impacts with our engineering, product and regulatory teams. After we have updated, reviewed and approved the new information in MatrixALM, the solution allows us to generate the new documentation at the touch of a button.”

Planning for the future

Cognoa plans to leverage additional capabilities of the Matrix Requirements solution to further improve its approach to requirements management. By enabling branching and merging for MatrixALM projects, the company will be able to gain a clear view of changes between releases faster than ever. And by enabling integration with GitHub, Cognoa aims to enrich its test results with additional data, facilitating root-cause analysis.

“One of our goals is to retain our start-up agility as we pivot into medtech—and MatrixALM helps us stay lean,” concludes Miller. “Thanks to the Matrix Requirements solution, we can help our engineers move quickly without compromising on compliance, and demonstrate to the FDA that we are following best practices every step of the way.”