Machine vision system checks complex automotive interiors

“Automotive manufacturers outsource their interior system requirements to maximize quality and reduce costs,” says Andrew Waller, Industrial Vision Systems Ltd. “Not only do such companies manufacture interior systems for the automotive industry, they design them, integrate them and deliver them to locations around the world. And, when your company is a Tier 1 supplier to one of the world’s most profitable automotive manufacturers, it is imperative that every product shipped is meticulously inspected.” 

IVS is no stranger to such tasks, providing automated inspection to many of the world’s top 100 Tier 1 suppliers to the automotive industry. “However”, says Waller, “unlike single components that may require a few simple visual or automated inspections to ensure such quality standards are maintained, automotive sub-assemblies such as car sub-assemblies may consist of a multiple variations of seat types all with different cloth or leather trim, headrests, seat positioning and motion handles. Such a large number of variants mandates the use of a number of different safety harness fittings, bezels and tracks which must be individually fitted to each seat before it is shipped to the automotive manufacturer. 

“Because the number of variants is so large,” says Waller, “it is relatively easy for a manual inspector to mistake one component, fabric or seat accessory for another.” Because of this, IVS was tasked with developing an automatic inspection system that would insure the correct placement of parts in the 26 different variants of car seat sub•assemblies produced by the Tier 1 supplier. 

“At first, the task of developing such a system may seem to be daunting,” says Waller. “However, when each individual task is broken down into a series of less complex tasks, such systems can be developed in a relatively short amount of time.” 

Along the line 

As with many automotive manufacturing systems, the assembly of car seats is based on a production line process. After each left and right side front passenger seats are fully assembled, they are placed onto a conveyor belt and move down the conveyor two at a time. 

To automatically inspect both seats simultaneously, IVS custom built an imaging work cell that uses multiple cameras, PC-based frame grabbers and lighting components all linked over a programmable logic controller to the Tier 1 supplier’s Supervisory Control and Data Acquisition (SCADA) system. This acts as a large-scale, distributed measurement and control system consisting of I/O hardware, PLC controllers, HMI, networks and database systems that is used to transfer data regarding each specific seat type to the machine vision system. 

As pairs of seat sub-assemblies enter the imaging work cell, a proximity sensor interfaced to the systems PLC triggers the presence of the part. Since the SCADA system is pre-programmed with the specific sub-assembly that should be present, it is used to open part-specific machine vision check routines. 

“For each pair of products,” says Waller, “forty individual checks are performed”. These include parts presence tasks, part type checks and a number of gauging applications. While the parts presence inspection software guarantees the correct placement of seatbelts, screws, under seat and side handles, parts type checking ensures that the correct fabric type, pattern and color has been installed. Finally, gauging functions ensure that the handles used to both move the seat back and adjust the seat position are of the proper type. 

“With a production cycle time of 80 seconds,” says Waller, “multiple cameras must be used to perform the inspection tasks. Because the seats must freely move through the imaging work cell, some of the cameras must be mounted on pneumatic gantries and moved within range of the seat sub-assemblies so that successful multiple inspections can be accomplished.” 

Multiple cameras 

To perform these tasks, no less than eighteen digital cameras perform parts presence, part type and gauging applications “In essence,” says Waller, “nine cameras are used on each car seat sub-assembly to perform these tasks. While seven cameras are positioned statically within the imaging work cell, two are mounted on pneumatic gantries and moved into position toinspect the head rest fabric and the fabric of the seat cushion. 

Each of the seven cameras that are statically positioned perform a specific machine vision task such as inspecting for the presence of the seat’s track end clip, wavy metal washers, outer side shields, screw types, and passenger handles. To provide the correct illumination for these cameras, several 6in white LED lights are used throughout the system. 

“As each sub-assembly enters the imaging workcell,” says Waller, “each of the LED lights is triggered from the system’s PC using a 64 channel digital I/O board that, in turn interfaces to the systems PLC. “In this way,” says Waller, “the lifetime of the LEDs can be extended since they do not need to constantly illuminate the seat sub-assembly.” 

Fabric inspection 

One of the most important tasks in the manufacture of such car seat assemblies is checking whether the correct fabric has been placed on both the car seat and the headrest. Since the placement of incorrect fabrics cannot easily be discerned by human operators, the system developed by IVS was also tasked with accomplishing this function. To properly inspect the fabric used in the assembly’s headrest, it was necessary to move the lighting to a position 5mm from the fabric itself. 

Similarly, to inspect the car seat fabric, the camera system was required to move to approximately 6in (150mm) from the fabric itself. This was accomplished by mounting each camera and lighting system on 200mm and 600mm stroke length, pneumatic cylinders, respectively. After the assembly enters the work cell, the cameras and lighting are moved into position and the fabric on the headrest and car seat illuminated. To light the headrest, a red on-axis ring light was used, while for car seat illumination, a White LED light was selected. 

“Selection of the correct lighting systems was crucial to the systems success, with the headrest presenting more of a lighting challenge due to the lack of pattern on the fabric in comparison to the seat , resulting in the use of close proximity on-axis lighting,” says Waller, “however, the inspection task that must be performed on both images is relatively similar. Indeed, in all the inspection tasks used in the system our software was used to perform all the parts, presence, gauging and fabric verification tasks’’ 

To perform fabric inspection after each individual image is digitized, a region of interest (ROI) is applied to the image which is then processed using a 9 x 9 erosion filter to enlarge the black objects within the image. This has the effect of increasing the intensity of the fabric’s weave. 

After this is accomplished, a Sobel edge detection filter is used to create horizontal white lines in the x direction of the image. “By binarizing this resulting image and performing a feature analysis function in the resulting image,” says Waller, “elongated objects in the x direction are highlighted and displayed.” By performing a template match with this image the resulting region of interest can be re-sampled into a fixed input size for use with a Neural Network Classifier. The pattern is checked against a trained Neural Network algorithm which has been created from a series of patterns of all of the possible fabric types. The Industrial Vision software then returns with a ‘Class Name’ match with a resulting percentage of certainty that can be evaluated against set acceptance levels. Based upon using this technique the software can discern whether the correct fabric has been used in the headrest build. 

Image processing 

Performing these image processing functions on different fabric types highlights the processes used to accomplish this task. “Of course,” says Waller, “a number of different image processing operations could have been used to produce a similar result. But by using the IVS graphical user interface, however, it was a relatively easy task to select and program a series of relatively simple machine vision functions to perform all the tasks required of the system in a less than 20second cycle time.” 

“Complex” says Waller, “This sophisticated multi-disciplined project was certainly made easier by implementing a common sense simplistic design philosophy, coupled with deployment of ‘off the shelf’ vision system peripherals, control systems and robust industrial vision software, to fulfill both the customers and our own design expectations” 

IVS software was also used in the development of a user-friendly operator interface that is used to display all the results of the 40 inspection tasks that take place. Should any specific task, such as the presence of a track-end clip fail, the specific function is highlighted in red. In such an event, the PC-based vision system transmits this information to the systems’ Micrologix  PLC from Allen Bradley. In turn, this actuates a set of pneumatically controlled cylinders that ejects the assembly from the imaging workcell. After re-work, the assembly can then be placed back on the production line for a further final inspection.