Tag Archives: hannah hager

Identifying Humans at Night with Face or Ear Recognition

In this interview, Dr. Thirimachos Bourlai, an expert at the forefront of face recognition technology, shares his opinions regarding the challenges of night time facial recognition systems. He also sheds light on the advantages and disadvantages of using visible or infrared sensors for practical facial recognition applications and scenarios. Finally, he briefly outlines the available modalities and developments in long-range human identification technology that can be used in the future to deal with problems of recognizing unfamiliar faces in still image and video biometrics.

Q: Could you give some details on the challenges in identifying humans at night with facial recognition?

A: Most face recognition systems depend on the usage of face images captured in the visible range of the electromagnetic spectrum, i.e. 380-750 nm. However, in real-world scenarios (military and law enforcement) we have to deal with harsh environmental conditions characterized by unfavorable lighting and pronounced shadows. Such an example is a night-time environment [1], where human recognition based solely on visible spectral images may not be feasible [2].

In order to deal with such difficult FR scenarios, multi-spectral camera sensors are very useful because they can image day and night [11]. Thus, recognition of faces across the infrared spectrum has become an area of growing interest [2-16]. Here is an example – realistic scenario – where it is very challenging to identify humans at night with facial recognition. Consider an individual walking towards (approaching the entrance of) a military facility. A face image needs to be captured by the surveillance cameras covering the facility that can be used for identification. The main challenges are (i) Data management: the raw, relevant (when the human is within the field of view), video footage (that can be several Mbytes or TBytes per day) needs to be narrowed down to information pertinent to the human and his/her face. At this point, applying efficient face tracking and eye detection techniques is very important [14, 15]. (ii) Data quality: improving the quality of the available face images [17], (iii) Face matching: applying state-of-the-art face matching techniques to perform identification [8, 16]. There is also another major challenge and that is face spoofing but this can be covered in another discussion.

Q: Could you frame the main challenges that are person-related, device-related and facial recognition software-based? Any other challenges?

A: There are various challenges with regards to night time facial recognition technologies. I would narrow the challenges down into four main categories:

1. Person-related: variations in pose, expression, including illumination that depends on the operational environment. For example, in certain night time scenarios there may not be sufficient ambient light to capture good quality photos. That is why the selection of sensors (e.g. infrared or other combinations) plays a very important role.

2. Device-related: using different camera sensors such as (i) cameras operating at different spectra vs. (ii) very expensive, high-end vs. low cost surveillance cameras.

3. Related to FR matching software used: (i) commercial software packages, in which the operator cannot access, know and/or change internal algorithmic functionalities (e.g. image restoration algorithms applied to raw data), vs. (ii) academic FR software packages, where the operators have access to the code and thus, can change, upgrade and improve the software (face matching) capabilities.

4. Related to other factors, such as image quality (e.g., image resolution, compression, blur), time span (facial aging), occlusion, and demographic information (e.g., gender, race/ethnicity, or age). For example, a face recognition system will behave differently when it is trained and tested using a certain cohort (such as a race group) or when using different cohorts.

Of course, the biggest challenge of all is the combination of the above challenges!

Facial recognition systems have come a long way in helping the military determine "who's there?" at night.
Facial recognition systems have come a long way in helping the military determine “who’s there?” at night.

Q: What are the advantages and disadvantages of using visible or infrared sensors?

A: This is a very challenging question mainly because it is very general but I am very happy that I can provide some insight into it. For example, someone could focus only on sensor technology (mainly hardware with sensors ranging from visible, Near Infrared and up to Long Wave Infrared) and someone else on spectral imaging and the information acquired from different sensors, including the software developed to process this information.

Regarding FR both visible and IR sensors are important. Visible sensors have the advantage that they are low cost and the spatial resolution can be much higher when compared to certain infrared sensors, especially short ware IR or cooled infrared (thermal) ones. The infrared (IR) spectrum is divided into different spectral bands based on the response of various detectors, i.e. the active IR and the thermal (passive) IR band. The active IR band (0.7-2.5µm) is divided into the NIR (near infrared) and the SWIR (short wave IR) spectrum. NIR has the advantage that we can see at night but the limitation is that an illuminator is required, which can be spotted (cannot covertly illuminate the scene). SWIR has a longer wavelength range than NIR and is more tolerant to low levels of obscurants like fog and smoke. Differences in appearance between images sensed in the visible and the active IR band are due to the properties of the object being imaged. The benefits of SWIR are discussed in [8]. SWIR may pick up facial features that are not observed in the visible spectrum and can be combined with visible-light imagery to generate a more complete image of the human face. The SWIR range has only recently become practical for FR, particularly since the development of indium gallium arsenide sensors, which are designed to work well in night-time conditions. Another advantage is that the external light source that may be required for regions in the SWIR band can covertly illuminate the scene since it emits light invisible to the human eye.

The passive IR band is further divided into the Mid-Wave (MWIR) and the Long-Wave Infrared (LWIR) band. MWIR ranges from 3-5µm, while LWIR ranges from 7-14µm. Both MWIR and LWIR cameras can sense temperature variations across the face at a distance, and produce thermograms in the form of 2D images. The difference between MWIR and LWIR is that MWIR has both reflective and emissive properties, whereas LWIR consists primarily of emitted radiation. The benefit is that they are both almost completely impervious to external illumination. Another advantage is that they reveal different image characteristics of the facial skin. However, their limitations are that they are subject to variations in temperature in the surrounding environment, and to the variations of the heat patterns of the face that can be affected due to various factors, e.g. stress, changes in temperature of the surrounding environment, physical activity etc. The importance of MWIR in FR technology has been recently and first proposed by MILab [10].

I would also like to redirect to some very interesting an important articles [18-22] and web-links [23-24]:

(i) Regarding spectral imaging, one of the most interesting and informative articles I have come across is that of Shaw and Burke [22], i.e. spectral imaging on remote sensing. Also, an interesting tutorial on infrared imaging can be found at [23], while another interesting link on night vision systems (low light imaging, near infrared and thermal imaging) can be found at [24].

(ii) Regarding face recognition and the comparison of visible against other infrared bands, the original work of Wilder et al. stands out [20], while some very interesting articles that came several year after can be found here [18, 19, 21]. Of course, MILab at WVU has recently published many articles in this area (details can be found in MILAB’s publications [1-16]).

Q: Could you provide us with a review of the available systems and developments in long-range identification? What are the contributions of MILab’s on long range FR?

A: There are many available camera systems used for long range biometrics and surveillance applications. There are commercially available products (including software) provided by different companies including L3, FLIR etc. There are also other systems (hardware and software) designed and developed under a specific research project (e.g. the TINDERS project [25]). Either type of systems has been used by researchers to perform specific biometric related experiments. At MILab, for example, we used two different types of infrared systems for long range night time FR, i.e. a NIR-based [9] and a SWIR-based [16].

In the first case [9], we used a NIR sensor designed with the capability to acquire images at middle-range stand-off distances at night. Then, we determined the maximum stand-off distance where FR techniques can be utilized to efficiently recognize individuals at night at ranges from 30 to approximately 300 ft. The focus of the study was on establishing the maximum capabilities of the mid-range sensor to acquire good quality face images necessary for recognition. For the purpose of that study, a database in the visible (baseline) and NIR spectrum of more than 100 subjects was assembled and used to illustrate the challenges associated with the problem. In order to perform matching studies, we used multiple FR techniques and demonstrated that certain techniques are more robust in terms of recognition performance when using face images acquired at different distances. In [9] you can find the details about the camera system (hardware) and the challenging FR experiments performed.

In the second case [16], we used a SWIR camera from Sensors Unlimited (bulk) or within a developed optical system that can perform long range imaging at day and night [25]. In that study, we investigated the problem of cross spectral FR in heterogeneous environments. Specifically, we investigated the advantages and limitations of matching SWIR (at 1550 nm) probe face images to visible (gallery) images acquired under variable scenarios: visible images were collected under controlled and semi-controlled conditions (full frontal faces, facial expressions, indoors and outdoors, short range, fixed standoff distance to 7 feet or 2 meters), while SWIR images were captured under (i) fully controlled indoor conditions; (ii) semi-controlled conditions (full frontal faces, indoors, long ranges, i.e., up to 348 feet or 106 meters); and (iii) uncontrolled conditions (variable poses, face expressions, occlusion, outdoors, night and day, variable range, i.e., up to 1312 feet or 400 meters). Three different matching/encoding algorithms were utilized, namely, Local Binary Patterns (LBP) and Local Ternary Patterns (LTP) [26], and a commercial face matcher. Our experimental results indicate that our proposed methodology (i.e., using a cross-photometric score level fusion scheme) performs better than baseline (single matchers before photometric normalization) cross-spectral FR performance, in the most challenging (uncontrolled) scenario described above. In [16] you can find the details about the camera system (hardware) and the challenging FR experiments performed.


Top 5 Threats to U.S. National Security in 2014

The U.S. military, intelligence services, FBI and numerous other organizations exist to protect national security in America. Countless budget dollars every year are poured into this endeavor and it is the one thing that is truly a non-partisan issue in Washington. Politicians may at times disagree about the approach, but all Americans agree on the need for a sound, competent, and effective national security policy. In order to determine which policies should be implemented and how dollars and resources are best used to achieve this task, it is fundamentally important to understand what threats exist.

Detailed are the top threats to the U.S. Intelligence Community, and therefore, U.S. National Security, as detailed by James R. Clapper, the Director of National Intelligence in his statement for the record at the Senate Select Committee on Intelligence in January 2014.

Cyber Attacks

In the past several years, many aspects of life have migrated to the Internet and digital networks. These include essential government functions, industry and commerce, health care, social communications and personal information. Russia and China hold a divergent viewpoint on the nature of state sovereignty in the global information environment and states’ rights to control the dissemination of content online, which have long forestalled major agreements.

Computer network exploitation and disruption activities such as denial-of-service attacks will continue. Further, destructive attacks that delete information or renders systems inoperable will increase as malware and attack tradecraft proliferate. The biggest threats are posed by:

Russia – The nation seeks changes to the international system for Internet governance that would compromise U.S. interests and values.

China – In the future, the nation will revise its multi-stakeholder model Internet governance while continuing its expansive worldwide program of network exploitation and intellectual property theft.

Iran and North Korea – Their development of cyber espionage or attack capabilities might be used in an attempt to either provoke or destabilize the U.S.

Terrorist organizations – Terrorists are gearing up to develop offensive cyber capabilities.

Cyber criminal organizations – Their main motivation is profit and they will continue to pose substantial threats to the trust and integrity of global financial institutions and personal financial transactions.


Threats posed by foreign intelligence entities through 2014 will continue to evolve in terms of scope and complexity. The capabilities and activities through which foreign entities – both state and non-state actors – seek to obtain U.S. national security information are new, more diverse, and more technically sophisticated.

Insider Threat/Unauthorized Disclosures
Trusted insiders with the intent to do harm can exploit their access to compromise vast amounts of sensitive and classified information as part of a personal ideology or at the direction of a foreign government.

Priority Foreign Intelligence Threats

Attempts to penetrate the U.S. national decision-making apparatus, defense industrial base and U.S. research establishments will persist. The leading state intelligence threats to U.S. interests in 2014 will continue to be Russia and China, based on their capabilities, intent and broad operational scope. They seek information on advanced weapons systems and private data from the energy, finance, media and defense sectors.

Terrorist threats emanate from a diverse array of terrorist actors, ranging from formal groups to homegrown violent extremists and ad hoc, foreign-based actors. The threat environment continues to transition to a more diverse array of actors, reinforcing the positive development of previous years. The diffusion of past powers has led to the mergence of new power centers and an increase in threats by networks of like-minded extremists with allegiances to multiple groups. Below are some examples.

Homeland Plotting
Homegrown Violent Extremists – U.S.-based extremists will continue to pose the most frequent threat to the U.S. Homeland. Insular HVEs who act alone or in small groups and mask the extent of their ideological radicalization can represent challenging and legal threats.
Al-Qa’ida – Despite sustained counterterrorism pressure and key organizational setbacks, among other things, al-Qa’ida is likely hoping for a resurgence following the drawdown of U.S. troops in Afghanistan in 2014.

Terrorist Activities Overseas
Persistent Threats to U.S. Interests Overseas – U.S. embassies, military facilities and individuals will face persistent threats in parts of South Asian, the Middle East and Africa.
Syria – Syria has become a significant location for independent or al-Qa’ida-aligned groups to recruit, train and equip a growing number of extremists, some of whom might conduct external attacks.
Iran and Hizballa – Iran and Hizballah are committed to defending the Asad regime and have provided support by sending billions of dollars in military and economic aid, training pro-regime and Iraqi Shia militants and deploying their own personnel into the country.

Weapons of Mass Destruction & Proliferation

Nation-state efforts to develop or acquire weapons of mass destruction and their delivery systems constitute a major threat to the security of the U.S., deployed troops and allies. The U.S. is focused on the threat and destabilizing effects of nuclear proliferation, proliferation of chemical and biological warfare (CBW)-related materials and development of WMD Delivery systems.

The top threats are from:

Iran and North Korea
Iran wants to improve its nuclear and missile capabilities while avoiding severe repercussions – such as a military strike or regime-threatening sanctions. They’ve made progress in some areas, including uranium enrichment, nuclear reactors and ballistic missiles, which strengthens the theory that Iran has the scientific, technical and industrial capacity to produce nuclear weapons.

North Korea
North Korea’s nuclear weapons and missile programs pose a serious threat to the U.S. and the security environment in East Asia, a region with some of the world’s largest populations, militaries and economies.

North Korea’s export of ballistic missiles and associated materials to several countries including Iran and Syria, and its assistance to Syria’s construction of a nuclear reactor illustrate the reach of its proliferation activities. The nation also recently announced its intention to expand its existing nuclear facilities to include uranium enrichment as well as develop a long-range missile technology that is capable of posing a direct threat to the U.S.


Threats to U.S. space services will increase in 2014 and beyond as potential adversaries pursue disruptive and destructive counterspace capabilities. Chinese and Russian military leaders understand the unique information advantages afforded by space systems and are developing capabilities to disrupt U.S. use of space in a conflict. Chinese military writings highlight the need to interfere with, damage and destroy reconnaissance, navigation and communication satellites.

China has satellite jamming capabilities and is pursuing antisatellite systems. Russia’s doctrine emphasizes space defense as a vital component of its national defense.

Russian leaders openly maintain that the Russian armed forces have antisatellite weapons and conduct antisatellite research. Russia has satellite jammers and is also pursuing antisatellite research. Russia has satellite jammers and is also pursuing antisatellite systems.

Joining Networks: Joint and Coalition Tactics in Cyber Warfare

Lt. Col. Patrick King talks Mission Command.
Lt. Col. Patrick King talks Mission Command.

In this interview, Lt. Col. Patrick King, Assistant Director of Operations, Electronic Warfare, United States Air Force shares his tips for achieving necessary cyber defense tactics between joint and coalition networks as well as his best practices for establishing the security of a tactical network.

You were awarded the “Best in the Air Force” for “Info Operations Team of the Year” and “Electronic Warfare Team of the Year” in 2012. Could you shed some light into how your aviation and cyber warfare experience has equipped you with the skills to meet new challenges and exceed goals?

As the Air Force’s Information Operations Team Chief, in South Korea, my team was located within the Strategy Division of the 607th Air Operations Center. My responsibilities included managing the Electronic Warfare Cell, Influence Cell—or psychological warfare as it used to be called – and the Cyberspace Warfare Cell. Those three cells were responsible to not only getting our U.S. and Alliance messages disseminated out to the battle space during exercises, but also the planning effort against any adversary aggression on the peninsula. We were charged with attempting to influence—directly effect—enemy communications so that the adversary’s leadership could not hear, see or talk.

The aviation experience I have from flying the electronic attack (EA) COMPASS CALL aircraft really helped solidify my approach to Information Ops warfare; the EC-130 COMPASS CALL aircraft specializes in denying the enemy the ability to communicate, particularly in the Command & Control (C2) realm; i.e., leadership’s ability to talk with their forces and frontline troops’ ability to do their jobs.

Therefore, my background and understanding of the importance information, or lack of information getting to enemy forces and their leaders in this case, helped my team focus on the critical nodes of our potential adversaries’ modes of communication or ability to get information. We were able to shape cyber targeting in the Korean theater and played a pivotal role in the contingency planning in case of North Korean aggression towards the Republic of Korea.

What I felt made our team stand-out was the ability to use the latest nodal cyber analysis tools to vastly improve support to our U.S. and Alliance participants during exercises and military planning. For the first time we incorporated U.S. Cyber Command into our planning and exercise support, which drastically improved our ability to synchronize our information operations support to planning kinetic and non-kinetic strike packages. This new emphasis on cyber planning, using the most state-of-art capabilities available, as well as having U.S. Cyber Command’s involvement and expertise really allowed our efforts to meet new challenges and exceeded goals that many didn’t believe we’d be able to do—particularly in such a short period of time.

Could you share your daily duties as a crisis leader and on-site program manager of operations, projects and programs?

My daily duties include managing the flight operations of our EC-130H COMPASS CALL aircraft at Davis-Monthan Air Force Base in Tucson, Arizona. This involves the wide-ranging program management of aircraft system upgrades and daily aircrew training of 157 flyers. The crisis manager duties I perform incorporate the emphasis on safe flight operations and handling of any in-flight emergencies of aircraft I’m flying or helping other aircrews aboard the COMPASS CALL that are currently up in the air flying, to recover safely with malfunctions.

Probably the biggest crisis management I perform is giving support to our EC-130H aircrews and maintenance personnel deployed down range in Afghanistan, supporting Operation ENDURING FREEDOM. Our electronic combat group of COMPASS CALL aircraft has been continuously deployed to Afghanistan for more than 10 years. Thus, we’re constantly rotating new flight crews and aircraft into that theater of operations to support the on-going commitment and flights. Our efforts there are vital to supporting and providing electronic warfare (EW) to U.S. and Coalition air, ground, naval and special operations forces.

Since cyber defense is a team effort, what are some of your tips for achieving tactical networking between joint and coalition networks?

Well, you’re 100% correct that cyber defense is a team effort. Everyone must work diligently to ensure safe COMSEC (communications security) practices while continuing to upgrade and install the latest computer protections for networks and systems. Our ability to network with our sister-services, particularly coordinate and information-share, has drastically improved in the past three to four years. We’re much more equipped to be able to email and communicate with others in a secure environment. I’d say that we still have some hurdles to cross in order to make networking with our coalition partners’ on-par with the communications we have with our sister-services. Part of that problem resides in the difficulty of getting the systems our coalition partners use to talk to our systems. But again, some strides have been made here too.

Additionally, I believe that in order to have good cyber defense it’s important to have an effective cyber offensive capability; therefore, any intrusions or alerts can be handled in a timely manner and mitigate risks and exposure. But the bottom line is that cyber defense takes everyone’s effort; good cyber defense and practices must be emphasized and understood that it’s imperative to being able to continue to do our jobs if the balloon goes up (hostilities). Lastly, it must continue to be stressed that a large part of cyber defense rests with keeping upgrades, or patches, up-to-date and monitored.

The public and private sectors both struggle with insecure web environments. What are your best practices for establishing the security of a tactical network?

The same struggles the public and private sectors have in securing websites, or emails, and even on-line operations (especially Wi-Fi) we have too in the tactical spectrum. Again, proper COMSEC and protocol on computers is the best security. Make sure your computer systems filter spam. Maximize encryption. Don’t trust unsolicited email. Be leery of every mail and attachment. Install antivirus software and make sure it’s kept up-to-date. Also, install a personal firewall and make sure that’s up-to-date, too. The same phishing and social engineering techniques that pose a serious concern to the public and private sectors—pose the same risks to our networks from malware and identity theft.

The younger generation is very good at not answering phone calls from people (and parents!) they do not know or recognize. Kids, for the most part it seems to me, let those calls go to voicemail. However, our society (especially kids) is very quick—perhaps too quick—to jump on text and email messages.

It’s in everyone’s best interest to make a moment and check whom the message (especially email) is from, before opening it. The same goes for attachments and links within emails. If a friend has emailed a joke as an attachment and they’ve included all their friends on the email, your computer is more at risk to malware or a Trojan horse virus.

It comes down to being vigilant and taking the time to look at the risk factors associated with the web environment you’re operating in. VPNs provide a better level of security. Smartphone security has not kept pace with traditional computer security on measures such as antivirus, encryption and firewalls. Smartphones are getting better, though, with features now such as the ability to wipe the device clean remotely, or delete known malicious applications remotely, and inclusion of authentication features now, such as device access passwords.

What does the future of Mission Command look like to you?

The Mission Command future looks very bright and will only continue to grow in importance as our society (and the world) increase dependence on technology. The use of technology in our lives–and our reliance on technology–is only going to get bigger and bigger. With that reliance is societies’ dependence on networks, computers to function, and thus, maintain order. Our banking and infrastructure are big targets. We have to make sure it’s protected. Intelligence protection—safeguarding personal information, business secrets, banking accounts, etc., is paramount. Leaders must continue to devise, plan and execute strategy that protects the networks as our technology gets more sophisticated. Mission Command’s ability to focus and network amongst intelligence, technology and strategy experts is key to ensuring that our civilian infrastructure and military power will be able to defend against, and defeat, any cyberspace attacks in the future.

This interview was orginially published on IDGA.