Sensor-based food inspection and sorting

Line scan SWIR camera Lynx proves its mettle with complex imaging tasks

By Christiaan Fivez(1) and Raf Vandersmissen(2)
(1) Christiaan Fivez – Technical Director TSS of Tomra Sorting Solutions | Food. Tomra Sorting NV, Research Park Haasrode 1622, Romeinse straat 20, 3001 Leuven, Belgium. www.tomra.com
(2) Raf Vandersmissen – CEO of sInfraRed Pte, Ltd – a Xenics company. sInfraRed Pte. Ltd., Blk 28 Sin Ming Lane #06-143, Midview City, Singapore 573972, Singapore. www.sinfrared.com

Abstract

Food inspection based on sensors for the visual, shortwave infrared, UV and X-ray realms in connection with automated sorting and classification methods has become a major field of application for advanced image capture systems and infrared cameras such as those made by Xenics.

Ensuring quality, freshness and appearance of fruits, vegetable, grain and meat products by removing unwanted or dangerous materials before packaging and customer delivery according to accepted quality and safety standards is one reason for the widespread deployment of food sorting systems throughout the delivery chain. Improving yield, productivity, cost efficiency and competitiveness as well as cutting down on waste is another – which is growing in importance worldwide.

A good example how far automated food inspection has come today are the food sorting systems offered by the leading vendor Tomra Systems ASA, based in Asker, Norway.

Tomra, operating globally in 80 markets, is a multinational enterprise and technology pioneer employing a total of 2400. Tomra operates two main business entities: Tomra Collection Solutions is concentrating on reverse vending of used beverage containers, compaction and material recovery (which was the original business aspect at the time of the company’s founding in 1972). Tomra Sorting Solutions is engaged in recycling, mining and food inspection. All these activities are based on advanced sensor technologies and high-speed information processing regarding the material content, shape, size, color and geometry of the objects to be detected as well as their defect status and damage characteristics.

Sorting Technologies used by Tomra Sorting Solutions

A closer look at the various technologies deployed by Tomra in their food sorting product families reveals the broad variety and sophistication of the sensing and illumination systems and methodologies. Many of the technologies developed or acquired by Tomra are proprietary, especially in the realm of basic methods and procedures, and are well kept secrets. On the other hand, the company orders components and systems to be integrated in their sorting machines, such as the InGaAs sensors and cameras made by Xenics. To accommodate its customers in the very competitive food processing industries Tomra operates six well equipped food sorting test centers, the closest in Europe being located in Leuven, Belgium.

LEDs are used to detect subtle color differences in fruit and vegetables (and other products) in the visual spectrum. Laser light sources are installed in many Tomra sorting systems to enable quality categorizing according to color, structure and biological characteristics. Another aspect is to detect and remove contaminants that may have the same visual appearance as the regular food items, especially when sorting fruit and vegetables with their varying degrees of freshness and consumer appeal in regard to product consistency.

Laser illumination is a method that Tomra has extensively used and developed to detect and discern ‘extraneous vegetable material’ (EVM) such as nutshells, stems knuckles, etc., or ‘foreign extraneous vegetable material’ (FEVM) such as insects or mice in food sorting applications. Sorting under laser illumination based on the Tomra ‘Detox’ system detects unwanted ingredients such as aflatoxin by deploying the appropriate sensors to capture the extremely low light reflected by this toxic fungus. Another sorting criterion enabled by the proper illumination and sensor technology is the removal of food items that are missing a prescribed degree of chlorophyll/solanine content, which indicates insufficient freshness, or a type of produce that is not selected to be present in a particular sorting flow such as potatoes, peas or spinach.

Another advanced sorting technique developed by Tomra to a high degree of practicability is the deployment of infrared spectroscopy as a follow-up procedure to regular inspection tasks. Spectroscopic analysis of food items examines them in terms of their structural make-up and hidden defects by observing their spectral response to specific illumination wavelengths in the visual and SWIR realms (figure 1). For this spectroscopic analysis in SWIR, Xenics has developed a custom-made InGaAs detector for Tomra. This method delivers more information on the goods examined and it enables a more detailed classification in regard to quality and the state of freshness of certain food items and their

Food sorting system detects defects and inappropriate shapes of fruit and vegetables in the visual & SWIR realm

Figure 1: Food sorting system detects defects and inappropriate shapes of fruit and vegetables in the visual & SWIR realm.

‘Biometric Signature Identification’. BSI enables to identify slight, otherwise suitability for human consumption. This has led Tomra to the development of a technique that it calls unnoticed chemical and molecular differences in the inspected food items and it serves as a differentiator of their quality and suitability. This development has earned Tomra the ‘Excellence in Research and Innovation Award’ at the 2015 INC Congress.

Naturally, camera systems including the appropriate lenses, focusing provisions and data interfaces for further processing and statistical analyzing the measurements are very much at the center of Tomra’s sorting technology development. The SWIR line scan camera Lynx offered by Xenics takes a special place in this regard. It is used mainly for food examination at various wavelengths in the shortwave infrared section of the spectrum covering the range of 900 to 1700nm. For example, detecting foreign materials in frozen fruits and vegetable product streams can be done by determining the water content under specific infrared illumination. The method has proved to be very effective as well as relatively simple and straightforward to deploy.

In general, Tomra offers three pre-defined configurations of its automated sorting systems. They are usually tailored for specific requests in terms of the functionality that customers in the various segments of the food processing industry require.

An interesting future extension of the sorting technologies deployed by Tomra today points in the direction of using artificial intelligence (AI) methods to establish smart ways of discerning quality and suitability of food items and distinguishing between food and non-food objects in a variety of food processing streams. This would involve machines that can set up themselves and make judgments on quality and suitability based on a learning curve of a large number (millions) of previous judgments. This would also take into account changing levels of acceptable quality and appearance of food items such as fresh or processed produce. Based on complex algorithmic determination, this information is to be fed back to the sorting machine on a real-time basis to accommodate the prevailing standards of customer acceptance. In effect, this would overcome today’s more or less predetermined sorting decision making by categorizing food items as ‘Good’ or ‘Bad’ and replace or complement it by establishing a scale of acceptable quality levels – in the end saving large amounts of food items otherwise discarded.

High-speed InGaAs line scan camera

High-speed InGaAs line scan camera Lynx 512 CL

Figure 2: High-speed InGaAs line scan camera Lynx 512 CL.

With its path-breaking InGaAs line scan camera Lynx (figure 2), which was introduced in 2010 and is used by Tomra since 2013 in a continuously expanding range of sorting systems, Xenics has established new performance categories for resolution and cost efficiency in the shortwave infrared between 900 and 1700 nm. The Lynx is based on the proven linear sensor series XLIN-1.7. It provides high optical sensitivity and a broad dynamic range for industrial image processing and optical coherence tomography. A to D conversion is done at 14 bits. Integration time setting offers flexibility from 1 μs to several milliseconds. The Lynx is available with a CameraLink or GigE Vision interface.

Xenics offers the Lynx line scan camera in three different versions: Lynx-512 offers a line length of 512 pixels (at a 25μm pixel pitch) for machine vision and inspection tasks. It is well suited for high-speed scanning tasks at line rates of up to 40 kHz. Pixel rate goes up to 25 Mpixels/s. The small pixel format of 25 x 25 μm² (square) or 25 μm x 250 μm (rectangular) enables the optimization of low-cost and compact SWIR imaging systems. For industrial sorting applications in the SWIR realm, not involving spectroscopy, square pixels are preferred to create a 2D image of the correct aspect ratio when capturing moving objects. The Lynx-512 is operated without cooler and therefore consumes less power. The medium-resolution Lynx-1024 is well suited for demanding industrial applications. Its spectral coverage matches the absorption spectra of low-level photon emission and is less sensitive to emissivity changes in thermal measurements. Lynx-1024 offers a very small pixel pitch of 12.5 μm at a pixel rate of 50 MPixels/s.

The high-resolution Lynx-2048 provides a unique layout of 1 x 2048 pixels at a total array length of 25.6 mm. It is directed at high resolution inspection systems, mainly in semiconductor, solar cell or OLED display production lines. Line rate goes up to 10 kHz. The pixel rate is 25 MPixels/s.
The Lynx SWIR line scan cameras output their image data via Gigabit Ethernet (GigE Vision) or CameraLink interfaces. Camera control and parameter selection are provided through the same interface. Trigger inputs and outputs will synchronize image capturing with selected external events.
The Lynx platform is compatible with C-mount lenses. Optionally, it provides a spectrometer flange. Sensor output is pre-processed on-chip via two CMOS read-out ICs (ROIC) with five integration capacities selectable individually or collectively at runtime.

Sorting of frozen fruit and vegetables based on water content to determine foreign objects

Figure 3: Sorting of frozen fruit and vegetables based on water content to determine foreign objects such as plastic, cardboard, metal, etc.

In applications such as food sorting the Lynx SWIR line scan camera, is able to distinguish between fruit and vegetable and unwanted foreign objects such as plastic, glass, metal, or cardboard, which have to be removed before packing and processing (figure 3). The distinctive feature here is water content. In combination with a detector sensitive for the visible spectrum, also color defects and shape deviations can be detected.

Figure 4 shows the concept of a multi-mode food sorter equipped with visible-light CCD detector and InGaAs SWIR camera placed alongside the path of the inspected product inside a sloping chute.

Schematic setup of a sensor-based automated food inspection and sorting operation