Neural Nets Pinpoint Manufacturing Problems During Clinical Trial

In August 1995, Frank Casciani was working as a regulatory consultant with Advanced Tissue Sciences (ATS), a tissue engineering company in La Jolla, CA. Casciani was looking over interim data from the clinical trial testing the effective of ATS's Dermagraft tissue replacement to treat diabetic foot ulcers, a painful and dangerous condition that affects 3% of the US population. With Dermagraft, ATS hoped to significantly improve upon the rather low heal rate—about 33%—for diabetic foot ulcer patients treated with existing methods. But Casciani was having a hard time interpreting the results that the clinical trial was producing.

"Quite frankly I was baffled," he says. "The results seemed to indicate a great deal of unexplained variability in the effectiveness of the product and overall there was less difference between the results of the treatment group and the control group. This was not consistent with pre-clinical studies, the feasibility studies or our expectations." Casciani concluded that the inconsistency in Dermagraft's performance had to be linked with some inconsistency in its very complex manufacturing procedure "If we could control this inconsistency we could maximize the effectiveness of the prod-uct."

Fortunately for Casciani and ATS, the researchers had been assiduous in their collection of data regarding the manufacture storage and viability of each individual Dermagraft product. "We even knew the shelf position of each piece," says Casciani. Unfortunately, there were more than 50 parameters, which could conceivably impact product performance. Statistical analyses were simply unable to produce results from this huge pool of data that satisfactorily accounted for the differences in patient outcome.

This is when Casciani turned to Marc Parham of Parham Analysis (Bedford, MA), a firm specializing in the application of artificial intelligence to extract knowledge from clinical data. Using a variety of neural network analysis (NNA) programs, Parham was able to turn mounds of process data—even on processes with 50 or more inputs—into prescriptions for controlling critical inputs to affect a desired outcome. "NNA is suitable for any process with a lot of variable that needs to be made perfect," Parham said. "The pharmaceutical industry, where the requirements to demonstrate product efficacy and process control are very stringent, is naturally an industry that has shown a lot of interest in our services."

Parham took ATS's manufacturing and clinical trial data and using several artificial intelligence programs quickly whittled the initial list of parameters down to 14. Eventually, working closely with Casciani and ATS, Parham produced a predictive model which was 94% accurate in predicting patient outcome after the first two Dermagraft treatments. The model also produced critical limits on storage time, viability and protein content that could be used to maximize the effectiveness of the Dermagraft treatments.

Parham's neural net isolated three crucial factors for determining Dermagraft's success: the number of days a piece had been frozen, piece viability, and protein content. These are shown on the three axes.

While ATS could not use this data to change the procedures of the ongoing clinical trial, they were able to produce interpretations of the clinical data showing that a 50.8% heal rate in those patients treated with Dermagraft product met the critical limits produced by the neural net program, and well above the 31.7% healing rate of the control group. Out of a baffling, ambiguous mass of information, Parham's analysis produced not only evidence of the efficacy of Dermagraft, but also production and storage protocols that would assure maximum effectiveness. "I can't imagine how we could have arrived at the sort of prescriptive conclusions we did without the neural networks," says Casciani. "With the data from a feasibility study, expert use of an NNA can make a clinical study a no-fail situation."

How NNAs Work

Neural networks allow the analysis of a large number of variables in a process, and determine the interactions between all of them. After reducing the number of variables to an optimum number for purposes of predictive accuracy, a computer model of the data is created. This model is a huge formula that the neural net compiles as it goes through cases and "learns" the nature of the relationships among the variables and between the variables and the different observed outcomes.

Generally, data from a trial is divided up into two groups: the training group, about 75% of the cases, from which the program learns; and the testing set, which is kept apart from the set which generated the model and against that the model's predictions can be tested. The neural network's model can then be used to establish critical control points and tolerances in the process to assure the best possible outcome.

Neural net analysis processes huge amounts of data and determines interrelations among variables and between variables and outcome.

For NNAs to work best, as much information as possible about the process in question must be provided, which is another reason why the technology is a good match for the pharmaceutical industry. "Lots of ‘just-in-case' data is collected in a clinical trial but is never really used—information on the blood chemistry of patients, for instance. This data is often only used to make sure that the test group and control groups are sufficiently randomized on that parameter. With NNA, we can look and see if that parameter had any influence on patient outcome, and we can see where that parameter should be to give the best chance for a positive outcome," Parham says. "Clinical trials work by isolating one variable, whether the treatment or the placebo was administered to the patient and randomizing all the other variables between the two groups," he continues. "This assumes that we understand all the variables when we really do not. NNA does not isolate a single variable: it works by considering all the variables, defining the relationships between them, and establishing which variables have a significant influence on outcome. With the intelligent application of NNA technology, it is possible to look at all the variables with a high degree of accuracy. We have consistently turned up something unexpected—a critical control factor that no one anticipated would be important going into the trial. This is why this technology is so important: It produces results that are beyond the capabilities of human intuition and human reason."

Though Parham currently specializes in pharmaceutical applications, the possibilities for NNA are diverse. He has been working with neural net analysis for seven years and has successfully applied the technology to a wide range of problems from chemical reaction yields to dialysis membrane design to drug solubility and tablet formulation testing. "The processes developed with NNA work like Swiss clockwork. If you stay within the tolerances we develop, then you will achieve the desired result."

For more information: Marc Parham, president, Parham Analysis, 6 Ruben Duren Way, Bedford, MA 01730. Telephone: 781-271-0836. Fax: 781-275-5197.