PhUSE Wiki

Industry standards implementation is reaching a certain state of maturity and many companies have engaged on the path of developing suite of standard programs to speed up the process of producing derived datasets and outputs but how to get the benefits quickly and keep work challenging and rewarding for staff in the long run?

Although the need for standardisation and development of standard programs becomes now obvious in our organisations, development of standard programs do not directly belong to the critical path of life-science companies where submissions or important trial milestones often drag resources at critical times. Organisations often struggle assigning and keeping resources for the time-consuming task of development of standard programs and adapt their structure and strategy to accommodate this process and their pipeline of clinical tasks.

A number of stakeholders are involved who do not limit to the only sphere of biostatistics. It is important to understand who they are and what are their needs and desires in order to shape the strategy and define adequate processes with respect to standard program development.

*: Medical Writers, Medical & Science, Health-Economics, Pharmacovigilance, Regulatory, Marketing, etc.

**: Contractors, CROs, BPOs, etc.

It all usually starts where cross-functional standardisation groups are formed with the mandate to develop high-level specs including output mock-ups or brief description of derived variables and options. Such documents are used to get feedback and involve less technical but not less important functions in the process such as Medical Writers or Clinicians. These deliverables serve then as basis for development of more detailed and technical specifications (Requirements) that can be used for development and validation of associated standard programs.

From that point, organisations’ strategies will vary from one company to another according to their size, culture and overall outsourcing capability. Companies indeed try to balance internal and external resources involved in standard program development between dedicated support departments and clinical projects. Staff in clinical projects represent the end-users of the standard programs and are at the same time working on the critical path of drug development with limited time to spend on other activities.

Here are different organisation set-ups used in the industry where pros and cons are each time highlighted.

One department developing all standard programs for the rest of the organisation

Pros:

• Clinical Projects can focus on their trials.
• Consistency across all developed standard programs.

Cons:

• Huge training needs.
• Possible gap in Expectations vs. Final Product.

One department responsible for the corporate level and processes and each Clinical Project develops their own tools in addition

Pros:

• Global standards are addressed centrally
• The burden is shared across the organisation
• More people are also involved and can directly contribute

Cons:

• Lot of work will be needed to align each clinical project development.
• Clinical Projects may develop tools that could have benefited other Clinical Projects.
• Risk having unfinished or obsolete standard programs due to conflicting clinical timelines due to development dragging over a longer period.

Requirements written and reviewed in-house and development outsourced, final product delivered with review rounds in between.

Pros:

• Commit less internal resources from both dedicated support department and clinical project.
• Consistency across all developed standard programs

Cons:

• The review time can be much bigger than you planed if your outsourcing partner doesn’t know well your company standards, processes and expected quality.
• Requirements need to be much more detailed.
• Final product may look like a totally new tool the internal staff does not adhere to.

One department doing it in collaboration with representatives from the different Clinical Projects where prototypes are being developed.

Pros:

• Better data standards adherence
• Enhanced clinical staff involvement, feedback and buy-in
• Standards supported by prototype are exposed to feedback from output consumers
• Best existing programming concepts and tricks can be dragged from the clinical projects
• Limited training needed late on

Cons:

• Some of your best resource can be pulled out from some important clinical trial tasks.
• Clinical programmers can struggle to find enough time to spend on prototypes.
• Or at the opposite, it can be difficult for the programmers to leave the “creativity” loop.
• And timelines can be harder to predict.

From the different organisation set-ups described in previous section, 2 pattern of development arise. Sequential and Recursive. The challenge remains to find the right balance between keeping enough internal resources assigned to the clinical projects and involve end-users in the development of such tools so project standards can be considered further and end-users adoption can be enhanced through out the process.

Standard program development must follow a software development life cycle at the opposite of custom/one-shot programs that will be used once in a study. It goes from

• Requirements (what the program must do)
• Code (technical implementation addressing requirements)
• Program documentation (Technical description of code aiming at facilitating maintenance)
• User guide (Technical documentation aiming at helping end-users using the program)
• Test plan (Test cases checking that all requirements are met)
• Test report (Documentation of test case execution)
• User acceptance test (A number of scenarios are tested formally or informally before release)
• To Release in production.

Both sequential and recursive approaches can be adapted to the V-model that is widely used in software development so development and validation remain compliant.

Sequential

Requirements -> Program code and documentation -> Test plan -> Test report -> User acceptance test -> Release in production

It is easy to outsource but relies on detailed requirements and strong programming guidelines and well established program design practices. There is always the risk creating a toolbox that will sound totally foreign to in-house programmers in clinical projects that they will have to learn like a new application vs. using their own suite of undocumented and unvalidated of programs.

When outsourcing this, make sure you have an in-house programmer assigned as reviewer who will follow the development and will be able to become “super-user” on the given component and can take ownership for the success of its release inside the organisation. Requirements and programming guidelines need to be more detailed and examples of in-house developed “state-of-the-art” standard programs to refer as gold examples.

Recursive

Draft requirements + Draft code prototype -> Use as-is within one or more projects -> Refinement of requirements -> Upgrade of prototype into robust and standardised program + doc -> Test plan -> Test report -> User acceptance test (anecdotic) -> Release in production

A prototype-based approach will aim at getting as much feedback as possible from most senior programmers and users before formal implementation. It will also help the users to learn the program and create a feeling of ownership. Naturally programmers will see some components they have spent time in improving as their “babies” and will be first advocate and support for using them in the future.

The challenge here can be to leave the “creative” loop and call a given prototype “final” (for now) and ready to be upgraded to a standard program.

Small wins early will come quicker. Prototyping can almost replace some of the analysis work that would lead to design requirements (more fun for a programmer than going on the black board). Following this, the requirements can be written retrospectively and the whole V-model followed step by step ensuring full compliance. During this process the prototype is elevated to some more robust code following internal guidelines regarding standard program practices including documentation. During the prototyping period, programs can be used in the projects as guinea pigs and double programmed for the time being. A given double-program can be reused across studies if needed in order to minimize validation time till it is fully validated and released as a standard program.

“Real” test for standard programmes is when programmers use it in real settings (trial reporting, submissions, publications). The second real test is when study team (Medical Writers, Medical & Science, etc.) reviews the produced outputs and check that they fulfil expectations from the defined and agreed standard. You can have surprises there too and if some modifications are needed, it is always better to get this feedback early on before finalising documentation and testing.

In a recursive type of approach involving prototyping, it is crucial that individuals are made responsible and have the mandate to get each given components into production. This is to avoid prototyping dragging for too long or being left aside due to important clinical deadlines.

Companies rush into full-on standardisation, buying new technologies and developing expensive tools while they haven’t done much about it for years and could still get drugs on the market. The backbone of success is definition of standards and their alignment across departments while prototyping will give the opportunity to see how processes, structures and technologies will support the best way possible the flow from Protocol to Statistical Outputs. Best is enemy of good and intending to get such complex data involving different department functions into a streamlined workflow requires time and patience. A Small-wins-early approach would be more beneficial in the long run. And this applies too at the single level of standard program development involving mainly biostatistics where an iterative approach would enable to drag the know-how from your best programmers, get their buy-in and provide a rewarding work environment for them.

Another good example of prototyping comes from Google ® that for both gmail ® and google+ ® (to mention some of their more famous products) gave an account to all their employees for a period of time to get their feedback before releasing them to a larger audience. We always have clinical trials or submission supported by experienced programmers that we can use to give us feedback on our tools in real settings.

The goal is to gain efficiency and minimize repetitive programming and validation and not to develop software in the first place. Simpler components developed first and used as prototypes within clinical projects enables to get quick wins on the KPIs and valuable feedback first on both program design and defined standard program development process before embarking on the journey of developing extensive suite of complex, flexible, robust and documented standard programs.

Let’s remember that we work in the innovative industry and innovation should be promoted through out our work processes. Programming must remain challenging, fun and rewarding for programmers and we need to continue providing an environment where they can contribute and develop.