Process assessment with Automotive SPICE^®

What does an ASPICE assessment usually mean? You will normally encounter this if you work for a supplier that develops electronics for the automotive industry. Your customer expects your established workflow processes to fulfill certain criteria.

Using the ASPICE criteria as a guide, you can shape your processes in a systematic way. The quality of your processes can affect the reliability of your product. Surprised? Take, for instance, a situation where you cannot control the configuration of a major version of your product, and so your customer is not able to integrate your system into the car. The result: The car manufacturer has to delay production start purely because of the missing system. Reliable processes can therefore be critical to an organization! ASPICE provides you with fundamental process criteria based on industry experience.

The assessment is conducted in order to evaluate whether your processes reflect these requirements. During an assessment, the implementation of a workflow is evaluated based on the process requirements specified by ASPICE.

Professionals often feel uneasy when it comes to an assessment, and your customer may send a third party to rigorously evaluate your processes. Once the assessment is completed, the customer learns about the capability of the project being evaluated. In turn, you benefit from the report of an ASPICE assessment, providing insights for process improvements.

An ASPICE assessment is an evaluation of the capability of your R&D processes.

Automotive SPICE is derived from the acronym for SPICE: Software Process Improvement and Capability Determination. The last letter represents dEtermination—the capability level achieved by your processes. Ultimately, ASPICE is a process assessment model to determine the capability level of your R&D processes.

Since your customer is the party calling for the assessments, the evaluation as a whole will consist of a selection of processes that are relevant to your customer’s products. The capability level relates, therefore, to these processes—generally speaking, ASPICE assessments do not generally cover a company’s entire R&D activity. In other words, if you work for car manufacturer A, the assessment will only cover the specific processes that are deployed in projects for manufacturer A. Projects for manufacturers B and C will not be assessed.

ASPICE only involves assessments, and not audits. In other domains, the difference between the two is debated: An audit is directed at the process and concerns the management system. The evaluation of an audit follows binary logic, determining only whether the requirements are met or not.

In an ASPICE assessment, your processes are examined based on actual evidence. You must demonstrate to the assessor through your work products that your processes meet the requirements. In the ASPICE assessment, the questions are based solely on the ASPICE requirements—and no additional internal process requirements are considered.

ASPICE takes a much more detailed approach than an audit: Instead of simply determining whether you pass or fail, ASPICE considers many parameters, starting with the so-called basic and generic practices that reflect proven industry practices. The process requirements that must be met in order to achieve the process goal (process outcomes) are listed. With your work products, you demonstrate that these practices are being implemented. The assessor is thus given an indication of process capability and can apply and evaluate the process based on process attributes.

The assessment thus indicates to what extent specific processes or their attributes meet the ASPICE requirements and reports on the maturity level of each assessed process. Based on a matrix, this report consolidates the evaluations of basic practices, process attributes, and capability levels into a single, comprehensive overview. It ultimately provides a detailed picture of how targeted the R&D processes of the project are and then identifies areas for improvement. This is what makes the ASPICE assessment a valuable tool.

An ASPICE assessment rigorously examines a range of processes within a specific project, identifying potential for process improvements.

What is the purpose of your ASPICE assessment? Are you looking to establish a foundation for process improvements, or aiming to identify process risks that could impact a particular version? An ASPICE assessment supports both goals, but the approaches differ.

Assessments for process improvement reveal strengths and weaknesses in the processes. In other words, they examine whether the processes are fit for their intended purpose. This information allows you to reshape your processes to eliminate weaknesses. You can utilize this type of assessment yourself during your process improvement phase and to prepare for a product risk assessment.

• Are you able to effectively address quality risks? Finding this out is the goal of a product risk assessment. When it comes to a product launch, the assessment represents the customer’s perspective, who wants to receive a reliable product on time.
  Practical example: SWE.6 is the software qualification testing process. In a process improvement assessment, we review your implementation of the process on the basis of the ASPICE process requirements. To do so, we examine relevant examples. Testing one-fifth of the requirements is sufficient for this purpose.
• If only 20% of the requirements for a version were tested in a product risk assessment, there would be a high risk that the software may not function properly.

Irrespective of whether your assessment is related to a product improvement project or product risks, the assessment focuses on capability in terms of the relevant processes.

Since it is independent individuals who will be assessing a number of processes in a specific project handled by your team, the structure of the assessment will reflect the interaction between these external parties and you and your team members. In the video and the free white paper, you will see a simple example outlining the typical phases and activities of an assessment process. Responsibility for some actions falls to the assessor, while other actions fall to your team and your organization. You can also differentiate between activities carried out on-site and those conducted elsewhere.

There are three phases of the assessment process:

1. Preparation phase
2. Execution phase
3. Closure phase

In the preparation phase, the focus is primarily on planning for the assessment. The scope of the assessment is defined, along with the goals and other factors that affect the assessment. The result is an assessment plan containing all the necessary information for the assessment, along with an assessment agenda that is agreed by the lead assessor, the sponsor, and the organization being assessed.

A central part of this is defining the assessment scope, which mainly includes processes, capability levels, instances, product components, teams, and locations.

An important decision that forms part of the planning phase is to determine the use case for the assessment. Are we talking about

1. an assessment for process improvement, or
2. a process-related product risk assessment?

My first video explains these different use cases in more detail.

In the preparation phase, there are four activities:

1. Initialization
2. Planning
3. Document review
4. Team preparation

The assessor moderates the introductory meeting with the sponsor and the representative of the project to be assessed. Planning the assessment process and reviewing documents are important preparatory actions for the lead assessor. As team leader, you should guide your project team. How to best do this is the subject of another video.

An assessment is most efficiently conducted on-site and in person. The assessment typically begins with a short opening briefing, during which the lead assessor explains the concept of ASPICE, the assessment process, and the scope of the assessment, in a short presentation. A representative of your company then introduces the product, the project, and the organization, after which

the interviews and document reviews are conducted according to the agenda. The actual interviews relating to the processes start with

1. Project management
2. Quality assurance, and
3. Configuration management.

These critical processes provide the assessors with valuable insights into the project. After this, the engineering processes follow in their customary order. The order is important, as the processes build upon one another. For example, in order to verify the traceability of system tests relating to the system requirements, we must first understand the structure of the system requirements.

The engineering processes are followed by the remaining support processes and supplier monitoring.

In between the interviews, the assessment team convenes for consolidation sessions, in which the evidence is reviewed and consolidated, and practices and process attributes are assessed. These consolidation meetings and their results are confidential. The main reason for this is that the assessments may need revision at a later stage due to new evidence.

At the end of the assessment, the assessors present the results.

In the closure phase, an assessment report is produced. This report is intended to provide your team with information about 

• the capability levels of your processes, which are individually evaluated based on specific parameters. The assessor calculates these using an assessment formula. The report includes the rationale behind the assessor’s rating decision. This involves justifying the impacts—or lack thereof—of relevant process outcomes.
• an explanation of how the presented evidence relates to the process requirements of ASPICE. A detailed description of identified strengths, weaknesses, and observations for each process activity
• the connection between the observed evidence and the documented observation
• a management summary identifying the key aspects of the assessment.

If you would like to know more about the formal process of an assessment, you will find a comprehensive overview in the Automotive SPICE guidelines.

With these three phases, you have now familiarized yourself with the structure of the ASPICE assessment process—from preparation to execution and reporting.
As with any process, an assessment involves various roles. I would like to talk about these briefly with you now.

There are three main roles: Lead assessor, co-assessor, and sponsor.

Let’s start with the lead assessor. This person is responsible for the procedurally correct execution of the assessment. A lead assessor must have the qualification of Automotive SPICE Competent or Principal Assessor Level.

The lead assessor is the individual who

• Communicates with the assessment sponsor
• Plans the assessment
• Ensures that the assessment team is qualified and that it covers all necessary aspects for the context of the assessment, such as Cybersecurity or functional safety
• Ensures that the assessment is conducted in accordance with the requirements of the assessment process
• Ensures that the assessment is properly documented.

An assessment team consists of at least two assessors: the lead assessor and one or more co-assessors. They must be certified by the VDA QMC; otherwise, the assessment may not be recognized by most OEMs. In an independent assessment, the lead assessor, at least, must belong to a different organization.

The sponsor of an assessment is responsible for ensuring that

• The assessors have valid certification
• The assessment team has access to all the resources they require.

The sponsor is the person who commissions the assessment, receives the assessment report, and decides to whom it will be forwarded. You can think of your sponsor as a sparring partner. They are an outsider to your project but highly invested in its success. If the sponsor is from your own company, they will typically be a member of the executive management. When a manufacturer commissions an assessment for a supplier, the representative of the supplier becomes the sponsor.

In addition, it has proven effective to appoint for each location an assessment moderator, who takes care of organizing the assessment.

In summary: For an assessment, you need individuals for at least three key roles: the lead assessor, one or more co-assessors, and the sponsor from your executive management.

The outcome of an assessment is of great importance for your project. You gain valuable insights into how to improve your workflows and reduce risks. It is therefore not helpful to manipulate evidence simply for the sake of the assessment. Consider the assessment as an opportunity to evaluate your current position and acquire useful insight.

Part of having the right mindset is preparing thoroughly. This is the only way you can reacquaint yourself with tasks and processes that may date back a long way. The assessment itself is subject to significant time pressure, but thorough preparation enables you to respond clearly and precisely. This allows you, to some extent, to have control over the assessment.

It is also helpful to familiarize yourself with the ASPICE terminology, so that you know immediately what the assessor is looking for when they are asking specific questions about a work product. After all, it is a bit dumb to have what you need, but not know what it even is, right?

The preparations begin with setting the assessment agenda in a timely manner. This is a prerequisite for identifying the right individuals to be the interviewees.

This phase is about understanding the current status of your project in light of the ASPICE requirements. Consider which areas in the project are critical, or areas that have been particularly challenging for your team. Try to match up the ASPICE practices with your own project implementation. If they don’t match up,

then identify the shortfalls between expectations and practice. Create an improvement plan detailing how you will address these gaps.

Carefully review what you and your team have achieved thus far. This allows you to address minor oversights—because nobody is perfect.

In the weeks leading up to the assessment, we recommend the following preparatory work:

• First of all, clean up the project folder. Ensure there are no issues with naming and versioning. Remove any artifacts that are no longer relevant.
• Ensure that all team members know where to find important documents.
• Prepare or select work products to present as evidence.
• Ensure that the interviewees understand the ASPICE requirements.
• Explain to them their role in the assessment, and
• guide them on how to explain how their tasks contribute to fulfilling the ASPICE requirements.
• The last point is to do with soft skills: You should thoroughly train your team members and representatives in your company on how to conduct themselves in the interview situation.

From this list, you will no doubt see how crucial thorough preparation is. You don’t want to make any mistakes, but you also don’t want to undervalue your performance.

The best preparation for an interviewee is to be very familiar with the content of the processes that are relevant to them.

• Do your team members know the associated process descriptions and steps?
• Are they familiar with the generic practices (GPs)? The GPs address aspects of capability levels 2 and 3, such as competence management and resource management.

What about the structure of the project folder?

• Do your team members understand the overall structure?
• Do they know where the work products are stored?
• Can they find them again quickly and easily?
• Do they know which work products and samples they want to present? Consider creating a list of work products as part of your preparation.

Do you speak ASPICE?

• Are you and your team members familiar with the key terminology of the standard?
• Do you understand these terms, and the underlying logic of ASPICE? Read the basic practices and generic practices that relate to all your relevant processes.

This briefing is essential, as interviewees are often unaware of where their work and performance fits into the model. This can lead to assessments being given a lower rating than they deserve. Avoid this trap by thoroughly preparing for the interview. Have answers ready for a potential final question: What topics have the assessors not covered? What do you do particularly well; what are you proud of?