Immeasurable amounts of personal and potentially incriminating data are currently stored in the websites, apps, and social media platforms that people access and update daily via their devices. Those data can become evidence for citizens, governments, and businesses to use in solving real financial, employment, and criminal issues with the help of a professional information gatherer.
Many people think using their favorite Internet search engine is sufficient to find the data they need and do not realize that most of the Internet is not indexed by search engines. SEC487 teaches students legitimate and effective ways to find, gather, and analyze this data from the Internet. You'll learn about reliable places to harvest data using manual and automated methods and tools. Once you have the information, we'll show you how to ensure that it is sound, how to analyze what you've gathered, and how to make it is useful to your investigations.
This is an entry-level, introduction to open-source intelligence (OSINT) course and, as such, will move quickly through many areas of the field. You will learn current, real-world skills, techniques, and tools that law enforcement, private investigators, cyber attackers, and defenders use to scour the massive amount of information across the Internet, analyze the results, and pivot on interesting pieces of data to find other areas for investigation. Our goal is to provide the OSINT knowledge base for students to be successful in their fields whether they are cyber defenders, threat intelligence analysts, private investigators, insurance claims investigators, intelligence analysts, law enforcement personnel, or just someone curious about OSINT.
Throughout the course week, students will participate in numerous hands-on labs using the tools and techniques that are the basis for gathering free data from the Internet. More than 20 labs in this course use the live Internet and dark web to help students gain real-world confidence. You'll leave the course knowing not just how to use search features on a website, but all of the scenario-based requirements and OSINT techniques needed to gather truly important OSINT data.
SEC487.1: Foundations of OSINT
We begin with the basics and answer the questions "what is OSINT" and "how do people use it." This first day is about level-setting and ensuring that all students understand the background behind what we do in the OSINT field. We also establish the foundation for the rest of the week by learning how to document findings and set up an OSINT platform, and we discuss effective research habits for OSINT analysts. This day is a key component for the success of an OSINT analyst because without these concepts and processes in place, researchers can get themselves into serious trouble during assessments by inadvertently alerting their targets or improperly collecting data, making them less useful when delivered to the customer.
During the first half of the day we work through the pieces of the OSINT cycle to understand what our process might look like. Then we move into how law-abiding people use OSINT to get the data they need, be it parents trying to figure out if the person they want to hire as a child care worker is trustworthy, a person "googling" someone they are going out with on a date, businesses looking for information about a rival company, or law enforcement using social media and OSINT to capture criminals. We then move into how criminals use OSINT to target victims and conduct other attacks against people and their electronic data. We finish the morning with a review of documentation tools for creating MindMaps, compiling notes, creating timelines, and analyzing relationships within data.
The day continues by jumping into understanding threat profiles so that we can protect ourselves and infiltrate the places we need to gather the data our customers want. Recognizing that some students will be creating their own OSINT collection platforms, we move into what that may look like, covering topics such as the platform, operating system, networks, tools, and plugins. At the end of the day, students learn about the use of sock puppet (false accounts) and set up their own account for the labs in the course.
CPE/CMU Credits: 6
SEC487.2: Gathering, Searching, and Analyzing OSINT
OSINT data collection begins on day two after we get a glimpse of some of the fallacies that could influence our conclusions and recommendations. From this point in the class forward, we examine distinct categories of data and think about what it could mean for our investigations. Retrieving data from the Internet could mean using a web browser to view a page or, as we learn in this section, using command line tools, scripts, and helper applications.
Our focus for day 2 begins with creating an OSINT assessment process and examining several OSINT frameworks that will help us while conducting our assessments. We then move to harvesting data from and about websites. Analyzing data such as SSL/TLS certificates and Google analytics IDs can be important to our assessments. Shifting from systems to people, we examine different methods for finding and validating basic data about people such as home addresses, phone numbers, and email addresses. These pieces of data become "pivot points" for our investigations, as we can often perform additional searches using these key data points to discover additional data that may be useful in our work. Students learn how to harvest user names and avatars and how they tie a single user to multiple user profiles across sites.
With user avatar images fresh in our minds, we pivot and consider how to perform reverse image searches. Day two concludes with a deep look at how we can execute advanced search engine queries to increase our chances of getting meaningful results.
CPE/CMU Credits: 6
SEC487.3: Social Media and Geolocation
Finding data on people, especially basic content such as email addresses, home addresses, and phone numbers, can be made easier using online people search engines. This is how day three kicks off, examining free and paid choices in this data aggregator area and understanding how to use the data we receive from them. Some of these engines provide social media content in their results. This makes a terrific transition for us to move into social media data.
The first social media site we look at from an OSINT perspective is Facebook, with its worldwide reach. Students explore Facebook profiles, groups, events, and other Facebook objects using graph searches and Facebook query techniques. We then move to detailed examinations of LinkedIn, Twitter, and Instagram, and what OSINT data can be found in each of them.
An increasing number of social media sites allow users to geolocate themselves. The afternoon of day 3 starts with an examination of how to harvest and use this content for OSINT. Then, focusing on the "social" aspect of social media, we dive into the content on dating and adult websites. A natural progression from dating is sometimes a wedding, so we inspect wedding websites and registries for OSINT data. Next, we see how we can use web and traffic cameras for remote reconnaissance. We finish the day by examining document and image metadata to glean interesting data points from different document types.
CPE/CMU Credits: 6
SEC487.4: Imagery, Networks, Government, and Business
Day four focuses on many different but related OSINT issues. We begin by looking at how various mapping sites can assist our assessments with aerial data, distance-measuring, and "street view" imagery. Moving beyond using just one vendor's mapping system, students will work with a variety of free, online mapping resources.
We then shift from OSINT about people and locations to OSINT about networks and computers, as researching IP addresses, domain names, and related content can be important aspects of our investigations. Starting with the basics, we get comfortable retrieving information about IP addresses, network blocks, and using the whois protocol. Students then move to making advanced queries to the domain name system (DNS) to grab subdomains and other domain data. To complete our work looking at computers, we examine how we use wireless network data in our work.
The second portion of the day has three modules. The first covers OSINT framework tool suites. These tools can accelerate our OSINT research by very rapidly acquiring data about people, networks, hosts, and more. We examine three frameworks in-depth during class. The next module covers harvesting information from federal, state, and local government web pages within the United States. The public data on these sites can help us research people and businesses. Completing the day, we look at the methods that can be used to gather data about businesses.
CPE/CMU Credits: 6
SEC487.5: The Dark Web and International Issues
The entire morning of day five focuses on understanding and using three of the most popular dark web networks for OSINT purposes. Students will learn why people use Freenet, I2P, and Tor. Each network is discussed at length so that students don't just know how and why to use it, but also gain an understanding of how those networks work. With the Tor network being such a big player in the dark web, the course spends extra time diving into its resources.
The first module in the afternoon examines how blue teamers (cyber defenders) can use monitoring to receive alerts when data of interest appears on the Internet. We then shift our focus to data found on "paste" sites. These websites sometimes contain content such as user names and passwords of compromised user accounts, detailed network information about our target's systems, or just data that our customers need to know.
Considering that a big barrier to using non-English websites can be the language, students learn how to use techniques to translate content and search locally for relevant information in our international OSINT section. We also examine how to discover popular websites and applications used in foreign countries. Since we talk about international data and traveling around the world, our courseware finishes up with an examination of how we track transportation (planes, boats, cars, etc.).
We leave some time at the end of the day for a massive lab, the "Solo CTF," which helps students put together all that they have learned in a semi-guided walk-through that touches on many of the concepts taught throughout the week. Setting aside time to work through our OSINT process in an organized manner reinforces key concepts and allows students to practice executing OSINT process, procedures, and techniques.
CPE/CMU Credits: 6
SEC487.6: Capstone: Capture (and Present) the Flag
The capstone for the course is a group event that brings together everything that students learned throughout the week. This is not a "canned" Capture the Flag event where specific flags are planted and your team must find them. It is a competition where each team will collect specific OSINT data about a certain group of people. The output from this work will be turned in as a "deliverable" to the "client" (the instructor), and then the three teams with the most-complete work will present their research to the class for voting.
This multi-hour, hands-on event will reinforce what the students practiced in the Solo CTF the day before and add the complexity of performing OSINT assessments under pressure and in a group.
CPE/CMU Credits: 6
Who Should Take This Course
We found that the more efficiently you find useful information on the Internet, the more successful you can be in your work. Whether you are trying to find suspects for a legal investigation or candidates to fill a job requisition, gathering hosts for a penetration test or placing honey tokens as a defender, this class will teach you techniques that will aid you in your work.
While far from complete, the Open Source Intelligence (OSINT) topics in SEC487 will be helpful to:
What You Will Receive
This Course Will Prepare You To
This is a learn it-do it course where we examine a topic and then dive into a hands-on lab to reinforce the learning. The course has over 22 labs spaced across the first five days, followed by the final hands-on Capture-the-Flag challenge on day six. Check out the lab content below to get a feel for what you will be doing within our class virtual machines.