Open Source Intelligence Tools (OSINT): Unlocking the Web’s Secrets

4
42126
Open Source Intelligence OSINT
Jeff Ogden (W163), CC BY-SA 3.0 via Wikimedia Commons.

What is Open Source Intelligence (OSINT)?

In today’s information-driven world, knowledge is power. Gathering intelligence has traditionally been the domain of government agencies and large corporations. However, with the rise of the internet and the abundance of publicly available data, a new field of intelligence has emerged: Open Source Intelligence (OSINT). OSINT involves collecting and analyzing information from publicly accessible sources to obtain valuable insights. In this article, we will explore the concept of OSINT, its benefits, and its growing significance in various fields.

Obtaining the information doesn’t require clandestine effort; it is retrieved legally and meets copyright requirements. The raw data and information are then collected and analyzed to help understand something more clearly. The OSINT framework supports decision-making, assesses public perceptions, forecasts changes, and more.

Who Engages in Open Source Intelligence Gathering and Analysis?

Technically, anyone who knows how to use the tools and techniques to access the information uses the process. However, the process is used formally by the United States intelligence community, the military, law enforcement, IT security professionals, private businesses, and private investigators.

Initially, the Central Intelligence Agency (CIA) created the Open Source Center, which builds on the established expertise of the CIA’s Foreign Broadcast Information Service (FBIS). The Open Source Center collects, produces, and promotes OSINT.

What types of data and information are considered to be Open Source?

Open-source data and information are available in various places and are most accessible online. Examples include:

The Benefits of OSINT

  1. Cost-effective: OSINT leverages free and publicly available information, making it a cost-effective approach compared to traditional intelligence-gathering methods that often require substantial resources and specialized expertise.
  2. Timeliness: OSINT allows for near real-time information gathering. With the proliferation of social media and online news platforms, OSINT analysts can quickly monitor and analyze unfolding events as they happen.
  3. Broad Perspective: OSINT uses diverse sources to provide a wider perspective on a given subject. It enables analysts to access viewpoints, opinions, and data from various individuals and organizations, leading to a more comprehensive understanding of the topic.
  4. Enhancing Decision-making: OSINT supports evidence-based decision-making by providing timely and relevant information. It enables individuals and organizations to make informed choices, mitigate risks, and identify opportunities based on a comprehensive understanding of the situation.

This book explains processes and methods using Open Source Intelligence techniques to uncover information. It describes using specialized online websites, open-source software, and creative search techniques to find information. In addition, the book covers such techniques as how to find hidden social media content, website owner information, photo metadata, sensitive documents, and much more. It explains where to find free investigative software, the best browser extensions, mobile apps, and the best target search engines.

Applications of OSINT

  1. Cybersecurity: OSINT plays a crucial role in detecting and preventing cyber threats. By monitoring online forums, social media platforms, and dark web communities, cybersecurity professionals can identify potential vulnerabilities, track hacker activities, and proactively respond to emerging threats.
  2. Law Enforcement: OSINT has become an invaluable tool for law enforcement agencies. It aids in investigations, intelligence gathering, and combating organized crime. Social media platforms and online forums have been instrumental in identifying suspects, monitoring criminal activities, and preventing potential threats.
  3. Business Intelligence: OSINT provides businesses with valuable insights into market trends, competitor analysis, and consumer sentiments. By monitoring social media discussions, customer reviews, and industry publications, companies can make data-driven decisions, improve their products or services, and gain a competitive edge.
  4. Journalism: Journalists increasingly rely on OSINT to support their investigative reporting. It enables them to verify facts, gather background information, and uncover hidden connections. OSINT tools assist journalists in researching stories, finding leads, and providing reliable information to the public.
  5. Academic Research: OSINT is essential to academic research across various disciplines. Researchers can utilize publicly available data and documents to study societal trends, analyze public opinion, and conduct quantitative or qualitative analyses.

List of Open Source Intelligence Tools

The tools and techniques used in Open-Source Intelligence research go much further than a simple Google search. The following is a list of helpful, time-saving open-source intelligence tools. Note that most of the resources are free, although some have advanced features for a fee.

Email Breach Lookup

  • Have I Been Pwned? – Have I Been Pwned is one of the most well-known and widely used websites to check if your email address has been compromised in data breaches. It provides information about breaches and the extent of data exposure, allowing users to take necessary steps to secure their accounts. This is one of my favorite open-source intelligence tools because anyone can use it. Just enter an email address to tell you which breaches the email affected. In addition, it gives a brief overview of the breach.
  • BreachAlarm – BreachAlarm monitors various sources for data breaches and allows users to check if their email addresses or accounts have been compromised. It provides real-time alerts and notifications about breaches that may affect you.
  • DeHashed – DeHashed is a breach search engine that enables users to search for compromised data associated with email addresses, usernames, and domains. It also offers a subscription service for more advanced features and breach monitoring.
  • Hacked-Emails – Hacked-Emails provides a database of email addresses and usernames exposed in data breaches. Users can search for their email addresses to see if they have been compromised.
  • Snusbase – Snusbase is a comprehensive database of credentials that have been leaked from data breaches.
  • Identity Leak Checker—Hasso Plattner Institute provides a service called Identity Leak Checker that allows users to check whether their email addresses, usernames, or passwords have been exposed to known data breaches.

Fact-Checking Websites

Hoaxy – Hoaxy is an open-source search tool that visualizes the spread of articles online (no longer active).

Media Bugs – A media bug is something in the media (e.g., newspaper, magazine, radio, etc.) that is wrong and correctable. Mediabugs.org allows anyone to report a bug, and they will work with the media outlet to correct it. Use the site to help identify fake or incorrect news and look for correct versions.

PolitiFact—Politifact focuses on fact-checking journalism. It uses a helpful rating scale to rate statements made by journalists, political figures, and others. The scale runs from “True” to “Full Flop.” It is a helpful way to know who is telling the truth and who is lying. For example, just read through some of the statements Politifact makes. Then, you’ll quickly realize that much of what we hear from politicians isn’t true. The company won a Pulitzer Prize for its work.

SciCheck—SciCheck is a feature of FactCheck.org that evaluates false and misleading scientific claims made to influence public policy. For example, when a political figure says that “global warming isn’t real,” they make a false claim to sway public opinion. SciCheck evaluates such claims and explains whether or not they are true.

Snopes – Snopes.com is one of the oldest, best, and most well-known fact-checking sites. The free site researches everything from urban legends, rumors, myths, questionable photos and videos, articles, and claims made by public figures. Note: Snopes was recently caught posting copyrighted material from other news sites.

Verification Junkie – Verification Junkie is a collection of tools to help verify and fact-check information and assess the validity of eyewitness reports. This site has a collection of very cool tools worth digging into.

Hacking and Threat Assessment

Norse – Norse maintains the world’s largest dedicated threat intelligence network. It has over eight million sensors that emulate over six thousand applications. Their network gathers data on who the attackers are and what they’re after. Their home page shows a live attack map with real-time information on attacks.

Spyse—Spyse is a cyber security search engine that helps you find technical information about internet-based assets. It offers a wide range of helpful tools, including a subdomain finder, reverse IP lookup, port scanner, and DNS lookup.

The following tools exceed most major search engines’ typical Image Search options.

Current Location – Current Location is a neat map search tool that allows you to select a position on the map anywhere in the world. Then, you can view photos within a certain radius of your location. It checks photos posted to Instagram, Flickr, and 500px.

Image Identification Project – The Wolfman Image Identification Project is a search tool that uses algorithms to identify images. Drag and drop or upload an image into the search form, which will attempt to tell you what it is. 

TinEye – TinEye is a reverse image search that allows you to search by image and find out where that image is online. You can upload a photo, drag it, and drop it in the search box, or search using a URL for an image. The index has over 30 billion images.

Public Records (Property)

Searching for someone? Run an online People Search through BeenVerfied and search billions of records in just a few seconds. Find addresses, phone numbers, criminal records, and more.

I assume everyone is familiar with the information you can get from sites like Zillow, Trulia, Realtor.com, etc. Those sites are always helpful and should be a part of your investigative toolbox. The following tools provide various twists on property records searching and are worth checking out.

Melissa Data Property Viewer – This open-source intelligence tool lets you view property information for almost any property. Just enter a zip code to start. Then, you can zoom in using a map or satellite view down to a specific property. Click on a particular property to get public records such as the full address, owner name, resident’s name, value, year built, building and lot square footage, and more. This can be helpful when planning surveillance and ensuring you have the right home.

Emporis Building Search – The Emporis site allows you to search for buildings, companies, and design/construction images worldwide. Their free building search engine enables you to search by building name to find the exact location of a building. This can be useful if all you have is the building name and you must reverse-match it back to an address.

OSINT Search Engines

Google Correlate—Google Trends Correlate can help you identify search patterns that correlate with real-world trends and determine similar search patterns. For example, you could use the tool to find out if there is a link between new legislation and gun sales (at least in terms of the search volume related to the two topics).

Million Short – Million Short has an exciting approach to searching for information online. The interface is generally similar to a typical search engine. However, you can sort and filter the results in several ways. Examples include popularity, eCommerce, live chat, date, location, etc. Also, it automatically pulls out the top sites that usually occupy the top spots in any search result (e.g., Amazon.com, eBay, YouTube, etc.)

Shodan—Shodan is a search engine and network security tool for the Internet of Things (IoT). It finds devices on the Internet, such as web servers, webcams, appliances, traffic lights, and even power plants.

TalkWalkerAlerts – TalkWalkerAlerts is an excellent alternative to Google Alerts because, in addition to monitoring the web for specific keywords, it monitors social media, blogs, and forums. This can be helpful if you need to find mentions of someone’s name or business across the web.

OSINT Social Media Search Tools

Facebook Search Tools – A collection of helpful links to Facebook online help resources and various open-source search forms. Learn to run searches and queries far deeper than the standard website or app search.

TweetBeaver—If you’re researching a Twitter user, TweetBeaver is a great resource. Use it to download a user’s followers list, check if two accounts follow one another, and much more.

OSINT Tool Websites

Many websites are dedicated to aggregating tools and links similar to this page. Some are excellent resources worth bookmarking, while others are just dead links. Following are some of the better resources:

  • IntelTechniques—This is probably the best site in the industry. It has many resources, links, and online search forms. The author, Michael Bazzell, provides training, news, and industry information.
  • OSINT Combine – Various free TikTok, Instagram, Reddit, Snapchat search tools, etc. It also has a reverse image analyzer, data visualization tool, and more.

Transportation

Aviation
  • Airportia​ – Tracks flights, airports, and airlines worldwide
  • FlightAware​ – Flight tracker with live maps and search function
  • Freedar – Flight tracker that includes military aircraft and air traffic control audio
  • RadarBox24​ – Flight tracker with live maps and search function. Displays flight info by default in the map view

Software


HunchlyHunchly is an online evidence-collection tool for investigative professionals. The software records your online activity to speed up the research and discovery.

Maltego – Maltego is a software tool developed by Paterva. Law enforcement, forensic investigators, and security professionals use it to analyze open-source intel. It runs on Windows, Linux, and OSX. Investigators use the software to collect data and information from various sources and display them graphically. This helps reduce analysis time, make connections, and uncover leads.

SearchCode –

Surveillance Cameras

Earth Cam – Earth Cam is the gold standard for aggregating the world’s public live-streaming webcams and surveillance cameras in a sensible way.

Insecam – Insecam aggregate access to hundreds of surveillance and public web cameras worldwide. The cams are not “hacked” and do not include cams on computers, laptops, tablets, etc. The site has cams organized by various categories, but finding the area you’re looking for is tedious.

Transportation – Vehicles, Aircraft, Boats

Flight Radar 24 – Created by two Swedish aviation experts, this OSINT site allows you to view live flight tracking information around the globe. The free service tracks over 180,000 flights per day. The graphical interface shows an interactive map with moving plane icons. Hover over an individual plane icon to see the flight number. Click on the plane icon to see flight information and details on the aircraft itself.

Marine Traffic—This website provides global ship tracking intelligence. It is another excellent example of visualization of large amounts of data. Visit the site and see a world map with a million little arrows. Each arrow represents a marine vessel. Click on any individual arrow to see what type of boat it is, such as a tanker, cruise ship, etc. You can filter the map in several ways, such as by ship type, capacity, voyage, and more.

Nationwide Aircraft Registry – Search a registry of all planes registered with the Federal Aviation Administration (FAA). The registry contains information about the aircraft, the owner(s), airworthiness, and more. The only challenge is knowing the registration number (n-number) to run a query.

VINCheck—VINCheck is an online tool the National Insurance Crime Bureau provides. It helps determine if a vehicle has been reported stolen but not recovered. Enter the whole VIN number and click on the Search VIN button to run a search.

Username Check

CheckUserNames—CheckUserNames is a handy open-source intelligence tool that checks the availability of user names on more than 500 social networks. Their new site is Knowem.com, which adds additional services, including searching the USPTO trademark database.

Namech_k – This OSINT tool is a search engine that allows you to search domain names to see what is available and then go through the process of registering it or making an offer. In addition, it will check to see if a username is available on hundreds of social media networking sites. You can even have them sign up for you.

Virus Scanner

VirusTotal – VirusTotal is an online tool that scans a file or URL to see if it has malware. VirusTotal uses virus scanners and URL/domain blacklisting services. The site allows users to upload a file or provide a URL. Then, it will run a scan and provide results to the user. In addition, VirusTotal shares the file/URL with the security community.

Visual / Clustering Search Engines

These search engines take searching a step further by sorting and organizing results. This helps users narrow down particular areas to focus on. Some allow users to use visualization tools to analyze data and information differently.

Carrot2.org – Carrot2 offers a unique search approach as it organizes search results into topics and visuals. They categorize results into topics in three ways. One is a series of folders by topic. Second, it displays the topics in a color-coded circle diagram (very cool). The size of the individual pieces indicates the number of results within each topic. The third method is “Foamtree,” which arranges the topics into a diagram by topic.

Yippy – Yippy searches multiple sources and provides results like any search engine. However, it also groups the results by topic for easier sorting and viewing. It is not as visual as Carrot2, but it is still helpful in culling down the results. Note: The website redirects to DuckDuckGo.

Website Analysis

BuiltWith.com – If you need to gather the technical details on a particular website, try BuiltWith.com. This OSINT tool is a website profiler, competitive analysis, and business intelligence tool. You can use their free search engine to analyze the technology profile of a website. Just enter a domain name and click search to find out which content management system (CMS) or content framework a site is using, including widgets and plugins.

Challenges and Ethical Considerations

While OSINT offers numerous benefits, it also presents challenges and ethical considerations. Privacy concerns, source reliability, and information overload are some of the critical difficulties OSINT analysts face. Adhering to legal and ethical guidelines when conducting OSINT activities is crucial to ensuring the responsible use of information and respect for privacy rights.

Note: Most of these OSINT tools on this list will work on standard web browsers such as Google Chrome, Internet Explorer, Firefox, and Safari. However, there may be minor differences in how the site displays on the screen or on your mobile device. Some of the solutions are designed specifically to search the dark web. You’ll need to download and install the TOR browser in those cases.

Questions / Suggestions

If you have any suggestions for this list of Open Source Intelligence tools, please post a comment below.

Michael Kissiah is the owner of Brandy Lane Publishing, LLC, which owns and operates a small portfolio of websites, including eInvestigator.com. Michael created eInvestigator.com more than 20 years ago after working as a private investigator in the state of Florida. Since that time, he has become an expert at how to find information online and has written over 1000 articles on topics related to the investigation industry. In addition, he is the author of the "Private Investigator Licensing Handbook", available at Amazon.com.

4 COMMENTS

  1. Yes, I’m trying to find someone that’s recently moved. I know the state and his name but am not 100% sure about the town but I also have his phone number but it is in his ex-wife’s name(or he says). Which one do you think I need to use to get answers on this guy?

  2. Am surprised that you haven’t suggested using the Internet Archive – great tool for checking the content and status of a particular website in the past – its a very useful tool as it can provide access to information on people which appears to be no longer publicly available i.e. in the case of insurance fraud (my field) the archived websites of sports club websites have proven to be a fertile hunting ground to disprove or mitigate injury claims

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.