Tag Archives: military

Identifying Humans at Night with Face or Ear Recognition

In this interview, Dr. Thirimachos Bourlai, an expert at the forefront of face recognition technology, shares his opinions regarding the challenges of night time facial recognition systems. He also sheds light on the advantages and disadvantages of using visible or infrared sensors for practical facial recognition applications and scenarios. Finally, he briefly outlines the available modalities and developments in long-range human identification technology that can be used in the future to deal with problems of recognizing unfamiliar faces in still image and video biometrics.

Q: Could you give some details on the challenges in identifying humans at night with facial recognition?

A: Most face recognition systems depend on the usage of face images captured in the visible range of the electromagnetic spectrum, i.e. 380-750 nm. However, in real-world scenarios (military and law enforcement) we have to deal with harsh environmental conditions characterized by unfavorable lighting and pronounced shadows. Such an example is a night-time environment [1], where human recognition based solely on visible spectral images may not be feasible [2].

In order to deal with such difficult FR scenarios, multi-spectral camera sensors are very useful because they can image day and night [11]. Thus, recognition of faces across the infrared spectrum has become an area of growing interest [2-16]. Here is an example – realistic scenario – where it is very challenging to identify humans at night with facial recognition. Consider an individual walking towards (approaching the entrance of) a military facility. A face image needs to be captured by the surveillance cameras covering the facility that can be used for identification. The main challenges are (i) Data management: the raw, relevant (when the human is within the field of view), video footage (that can be several Mbytes or TBytes per day) needs to be narrowed down to information pertinent to the human and his/her face. At this point, applying efficient face tracking and eye detection techniques is very important [14, 15]. (ii) Data quality: improving the quality of the available face images [17], (iii) Face matching: applying state-of-the-art face matching techniques to perform identification [8, 16]. There is also another major challenge and that is face spoofing but this can be covered in another discussion.

Q: Could you frame the main challenges that are person-related, device-related and facial recognition software-based? Any other challenges?

A: There are various challenges with regards to night time facial recognition technologies. I would narrow the challenges down into four main categories:

1. Person-related: variations in pose, expression, including illumination that depends on the operational environment. For example, in certain night time scenarios there may not be sufficient ambient light to capture good quality photos. That is why the selection of sensors (e.g. infrared or other combinations) plays a very important role.

2. Device-related: using different camera sensors such as (i) cameras operating at different spectra vs. (ii) very expensive, high-end vs. low cost surveillance cameras.

3. Related to FR matching software used: (i) commercial software packages, in which the operator cannot access, know and/or change internal algorithmic functionalities (e.g. image restoration algorithms applied to raw data), vs. (ii) academic FR software packages, where the operators have access to the code and thus, can change, upgrade and improve the software (face matching) capabilities.

4. Related to other factors, such as image quality (e.g., image resolution, compression, blur), time span (facial aging), occlusion, and demographic information (e.g., gender, race/ethnicity, or age). For example, a face recognition system will behave differently when it is trained and tested using a certain cohort (such as a race group) or when using different cohorts.

Of course, the biggest challenge of all is the combination of the above challenges!

Facial recognition systems have come a long way in helping the military determine "who's there?" at night.
Facial recognition systems have come a long way in helping the military determine “who’s there?” at night.

Q: What are the advantages and disadvantages of using visible or infrared sensors?

A: This is a very challenging question mainly because it is very general but I am very happy that I can provide some insight into it. For example, someone could focus only on sensor technology (mainly hardware with sensors ranging from visible, Near Infrared and up to Long Wave Infrared) and someone else on spectral imaging and the information acquired from different sensors, including the software developed to process this information.

Regarding FR both visible and IR sensors are important. Visible sensors have the advantage that they are low cost and the spatial resolution can be much higher when compared to certain infrared sensors, especially short ware IR or cooled infrared (thermal) ones. The infrared (IR) spectrum is divided into different spectral bands based on the response of various detectors, i.e. the active IR and the thermal (passive) IR band. The active IR band (0.7-2.5µm) is divided into the NIR (near infrared) and the SWIR (short wave IR) spectrum. NIR has the advantage that we can see at night but the limitation is that an illuminator is required, which can be spotted (cannot covertly illuminate the scene). SWIR has a longer wavelength range than NIR and is more tolerant to low levels of obscurants like fog and smoke. Differences in appearance between images sensed in the visible and the active IR band are due to the properties of the object being imaged. The benefits of SWIR are discussed in [8]. SWIR may pick up facial features that are not observed in the visible spectrum and can be combined with visible-light imagery to generate a more complete image of the human face. The SWIR range has only recently become practical for FR, particularly since the development of indium gallium arsenide sensors, which are designed to work well in night-time conditions. Another advantage is that the external light source that may be required for regions in the SWIR band can covertly illuminate the scene since it emits light invisible to the human eye.

The passive IR band is further divided into the Mid-Wave (MWIR) and the Long-Wave Infrared (LWIR) band. MWIR ranges from 3-5µm, while LWIR ranges from 7-14µm. Both MWIR and LWIR cameras can sense temperature variations across the face at a distance, and produce thermograms in the form of 2D images. The difference between MWIR and LWIR is that MWIR has both reflective and emissive properties, whereas LWIR consists primarily of emitted radiation. The benefit is that they are both almost completely impervious to external illumination. Another advantage is that they reveal different image characteristics of the facial skin. However, their limitations are that they are subject to variations in temperature in the surrounding environment, and to the variations of the heat patterns of the face that can be affected due to various factors, e.g. stress, changes in temperature of the surrounding environment, physical activity etc. The importance of MWIR in FR technology has been recently and first proposed by MILab [10].

I would also like to redirect to some very interesting an important articles [18-22] and web-links [23-24]:

(i) Regarding spectral imaging, one of the most interesting and informative articles I have come across is that of Shaw and Burke [22], i.e. spectral imaging on remote sensing. Also, an interesting tutorial on infrared imaging can be found at [23], while another interesting link on night vision systems (low light imaging, near infrared and thermal imaging) can be found at [24].

(ii) Regarding face recognition and the comparison of visible against other infrared bands, the original work of Wilder et al. stands out [20], while some very interesting articles that came several year after can be found here [18, 19, 21]. Of course, MILab at WVU has recently published many articles in this area (details can be found in MILAB’s publications [1-16]).

Q: Could you provide us with a review of the available systems and developments in long-range identification? What are the contributions of MILab’s on long range FR?

A: There are many available camera systems used for long range biometrics and surveillance applications. There are commercially available products (including software) provided by different companies including L3, FLIR etc. There are also other systems (hardware and software) designed and developed under a specific research project (e.g. the TINDERS project [25]). Either type of systems has been used by researchers to perform specific biometric related experiments. At MILab, for example, we used two different types of infrared systems for long range night time FR, i.e. a NIR-based [9] and a SWIR-based [16].

In the first case [9], we used a NIR sensor designed with the capability to acquire images at middle-range stand-off distances at night. Then, we determined the maximum stand-off distance where FR techniques can be utilized to efficiently recognize individuals at night at ranges from 30 to approximately 300 ft. The focus of the study was on establishing the maximum capabilities of the mid-range sensor to acquire good quality face images necessary for recognition. For the purpose of that study, a database in the visible (baseline) and NIR spectrum of more than 100 subjects was assembled and used to illustrate the challenges associated with the problem. In order to perform matching studies, we used multiple FR techniques and demonstrated that certain techniques are more robust in terms of recognition performance when using face images acquired at different distances. In [9] you can find the details about the camera system (hardware) and the challenging FR experiments performed.

In the second case [16], we used a SWIR camera from Sensors Unlimited (bulk) or within a developed optical system that can perform long range imaging at day and night [25]. In that study, we investigated the problem of cross spectral FR in heterogeneous environments. Specifically, we investigated the advantages and limitations of matching SWIR (at 1550 nm) probe face images to visible (gallery) images acquired under variable scenarios: visible images were collected under controlled and semi-controlled conditions (full frontal faces, facial expressions, indoors and outdoors, short range, fixed standoff distance to 7 feet or 2 meters), while SWIR images were captured under (i) fully controlled indoor conditions; (ii) semi-controlled conditions (full frontal faces, indoors, long ranges, i.e., up to 348 feet or 106 meters); and (iii) uncontrolled conditions (variable poses, face expressions, occlusion, outdoors, night and day, variable range, i.e., up to 1312 feet or 400 meters). Three different matching/encoding algorithms were utilized, namely, Local Binary Patterns (LBP) and Local Ternary Patterns (LTP) [26], and a commercial face matcher. Our experimental results indicate that our proposed methodology (i.e., using a cross-photometric score level fusion scheme) performs better than baseline (single matchers before photometric normalization) cross-spectral FR performance, in the most challenging (uncontrolled) scenario described above. In [16] you can find the details about the camera system (hardware) and the challenging FR experiments performed.


It takes a community to combat PTSD

PTSD victims often suffer alone.
PTSD victims often suffer alone.

An alpha male is assertive. He’s dominant. He’s often found within leadership positions, and frequently ranked in the military. He’s expected to hide his emotions and deal with personal issues privately.

An alpha male is not expected to take his own life. But many are; and more and more frequently they are taking innocent bystanders with them.

Iraq war veteran Ivan Lopez opened fire April 2 at the Fort Hood Army base in Texas, killing three people and injuring 16 before turning the gun on himself. At the time of the mass shooting, Lopez was undergoing tests to identify signs of Post Traumatic Stress Disorder, or PTSD. None were found.

Among other tours, Lopez had served four months in Iraq in 2011, but was not wounded nor did he suffer any battle-related Traumatic Brain Injury, according to Army Secretary John McHugh.

This mass shooting – the second at Fort Hood in less than five years when U.S. Army Major Nidal Malik Hasan shot dead 13 people and injured 30 more in 2009 – compounds an issue that has been taboo for far too long within the U.S. military.

Mental illness amongst active duty military personnel and veterans is being swept under the rug. And, we’re all suffering because of it.

Many military personnel are unsuccessfully dealing with the side effects of multiple deployments, including separation from their families and support networks. When discharged, they struggle to acclimate to everyday civilian life.

They are human, but we expect them to be superhuman because of their rough-and-tough lifestyles.

Lopez was married and had a 3 year-old daughter. He saw no direct combat. But although he had no visible injuries, reports show that he had other, less apparent ailments: depression, anxiety and sleep disturbances. The same military doctors who did not find any PTSD symptoms in him, also did not find him unfit in a way that would have flagged a background check and prevented him from purchasing the .45-caliber Smith & Wesson handgun he used in the shooting.

My question is how many deaths and shootings need to occur before our military starts to weigh equally mental and physical wounds?

Retired U.S. Marine Corps Corporal Adam Shatarsky is co-founder of the non-profit, The Wounded Walk, which will host a rally April 12 in Columbus Circle to raise awareness for physically and mentally wounded veterans.
He describes the Marine Corps as a dog-eat-dog world where if a soldier has a problem, they are expected to deal with it on their own. It’s unspoken that you should just “man up.”

One of Shatarsky’s fellow Marines committed suicide while serving at Camp Pendleton in California. According to Shatarsky, there was no indication that he would take his own life. He hadn’t shown any signs. Time and again we hear that there was no indication and that “we never thought it would happen to us.”

The record amount of military suicides is a whisper until it’s replaced by the shout of a mass shooting.

But, last year the number of suicide deaths within the U.S. military hit a record high and exceeded the number of those who died in combat in Afghanistan in 2012. An average of 18-22 veterans commit suicide daily – totaling 8,000 per year, according to the 2012 Suicide Data Report released by the U.S. Department of Veterans Affairs.

The U.S. Department of Veterans Affairs offers outreach in nearly 200 specialized PTSD treatment programs throughout the country. These medical centers offer one-one mental health assessments and psychotherapy, medications and group therapy.

But many military vets consider the outreach programs to be “clinical” and “unwelcoming.” What’s not considered is that these men and women are trained to not only hide their emotions, but to not have emotions at all.

How comfortable, then, would they feel walking through the doors of a VA-sponsored medical center?

To their credit, the military and the VA are now more active in ever before in their campaigns to de-stigmatize PTSD, depression and anxiety. Several military personnel have taken the mental health issue into their own hands by offering innovative solutions such as yoga classes with the goal of teaching the soldiers how to counter their stress through meditation and other non-pharmaceutical methods.

This shooting shouldn’t launch another gun law debate. It should trigger discussions on what’s behind the gun – mental illness. The stigmas of weakness that have long been associated with depression, anxiety and other mental disorders need to be chipped away within the military and by us citizens as well.

Sufferers of mental illness may show no signs while they deal with the accompanying shame, secrecy, isolation and social exclusion. There is no shame in seeking help, but we as a community need to make it known that it is ok to do so by changing the conversation around mental illness.

Shatarsky is lucky. He has a support network in his fellow Marines. His goal is to create a family and a community where veterans come to share their stories – and their grievances.

We all need to make every man and woman feel so lucky. We as a community need to step up to create support groups for our wounded veterans. The goal should be to decimate isolation and depression and replace it with social inclusion and reduced discrimination. We need to let veterans know that they are not alone.

The nature of the stigma will evolve as the stigma is challenged: There is no weakness in mental illness.

Our job is to identify the stigma in every context – then destroy it.

This was published on CNN and the LoudounTimes.