Provided by SANS

About the course

Security Information and Event Management (SIEM) can be an extraordinary benefit to an organization's security posture, but understanding and maintaining it can be difficult. Many solutions require complex infrastructure and software that necessitate professional services for installation. The use of professional services can leave security teams feeling as if they do not truly own or understand how their SIEM operates. Combine this situation of complicated solutions with a shortage of available skills, a lack of simple documentation, and the high costs of software and labor, and it is not surprising that deployments often fail to meet expectations. A SIEM can be the most powerful tool a cyber defense team can wield, but only when it is used to its fullest potential. This course is designed to address this problem by demystifying SIEMs and simplifying the process of implementing a solution that is usable, scalable, and simple to maintain.

The goal of this course is to teach students how to build a SIEM from the ground up using the Elastic Stack. Throughout the course, students will learn about the required stages of log collection. We will cover endpoint agent selection, logging formats, parsing, enrichment, storage, and alerting, and we will combine these components to make a flexible, high-performance SIEM solution. Using this approach empowers SIEM engineers and analysts to understand the complete system, make the best use of technology purchases, and supplement current underperforming deployments. This process allows organizations to save money on professional services, increase the efficiency of internal labor, and develop a nimbler solution than many existing deployments. For example, many organizations pay thousands of dollars in consulting fees when a unique log source needs a custom parser. This course will train students how to easily parse any log source without requiring consulting services, saving their organizations both time and money, and facilitating faster collection and use of new log sources.

SEC455 serves as an important primer to those who are unfamiliar with the architecture of an Elastic-based SIEM. Students that have taken or plan to take additional cyber defense courses may find SEC455 to be a helpful supplement to the advanced concepts they will encounter in courses such as SEC555. In addition, the material discussed in this course will enable students to not only build a new SIEM, but improve and supplement their already existing implementations, producing a more efficient solution that provides the answers they need more quickly and at a lower cost. The overall goal is to educate students on what they need to know to design and modify a SIEM, improve upon their current solution, and enable them to reach their original defensive goal - catching adversary activity in their environment.

This course will prepare you for:

• Architecting and designing a SIEM solution
• Designing a SIEM focused on speed and efficiency
• Deploying an open source SIEM solution meant for enterprise workloads
• Sizing and using a SIEM based on any budget (from shoestring budgets to unlimited funding)
• Collecting and parsing logs of any type or source
• Scaling log collection, ingestion, and search capabilities
• Enriching logs to provide advanced detection as well as context to analysis
• Building a compliance and tactical SIEM, whether a single system or dual stack (multiple SIEMs)
• Knowing when, why, and how to deploy multiple SIEM solutions and how to integrate them
• Deploying an alert engine and setting up alert rules
• Implementing tiered storage with aging policies to handle data retention and disk speeds
• Enhancing logs to add context
• Implementing searches that do not take coffee breaks to finish
• Knowing when and when not to augment logs
• Finding meaningful log sources and how to automate data collection
• Identifying common SIEM deployment pitfalls and hurdles

Course Syllabus

SEC455.1: Distributed Search and Visualization

Overview

Day 1: 9:00AM - 6:00PM

Day one focuses on Elasticsearch and Kibana and will take students on a journey from their first steps in the Elastic stack, to having a secured and production-ready Elasticsearch and Kibana instance by the end of the day. Students will learn the skills required to install, configure, and use Elasticsearch, and will become comfortable with using Kibana to visualize imported data in multiple useful ways.

Class begins with an introduction to the components of a SIEM and how each relates to the pieces of the Elastic stack. After a quick, high-level view, Elasticsearch receives a deep dive with a focus on the core practical concepts of node types, indexes, shards, and data type mapping. Also, administrative activities such as cluster creation, management, data retention and optimization are covered and put into practice with hands-on labs. Through these activities, students will become comfortable creating, modifying, and managing their Elasticsearch cluster. The Elasticsearch lesson also includes recommendations and calculations to ensure the capacity of the cluster meets storage and event-per-second requirements.

The second part of the day features a similar deep dive on how to install, setup, and use Kibana. Students will become familiar with the search, visualization, and dashboard interfaces, and will learn how to use these tools to explore log data. Also, students will learn how to secure access to their Elastic stack and to lock down indexes and documents with role-based permission schemes.

Exercises

• Installing and Configuring Elasticsearch - The first step in our SIEM, installing Elasticsearch from scratch and configuring it to be production-ready and usable for the rest of the class.
• Cluster Creation and Management - Setting up Cerebro for cluster and index management, creating an Elasticsearch cluster, using index templates for routing and understanding node failover.
• Kibana - Learning how to use the Kibana interface to run searches, and create visualizations and dashboards, then using them to explore and answer questions about your data.
• Securing the Stack - Using X-Pack to enable Authentication, Authorization, and Auditing, as well as encryption between nodes and to Kibana

CPE/CMU Credits: 7

Topics

SEC455.1 Distributed Search and Visualization

• What is ELK?
• Elasticsearch
  • Indexes, Shards and Replicas
  • Fields and Mappings
  • Node Types
  • Clustering
  • Cerebro
  • Curator
  • Hot / Warm Architecture
  • Data Retention and Optimization
  • Backup
  • Hardware Sizing
  • Index Mappings
  • Data Types
  • Templates
• Kibana
  • Searches, Filters, and Wildcards
  • Adding Index patterns
  • Linking to data from logs
  • Visualization Types
  • Aggregations, Bucketing, and Metrics
  • Creating Visualizations
  • Creating Dashboards
  • Timelion
  • X-Pack Tools
  • Graph Analytics
  • Machine Learning
• Securing the elastic stack
  • Security Plugin Options
  • Authentication, Authorization, and Auditing
  • Encryption and IP Filtering

SEC455.2: Enriching and Managing Logs

Overview

Day 2: 8:00 AM - 5:00 PM

Building on the infrastructure prepared during day one, day two focuses on how to efficiently move logs from your edge devices, and then transport, parse, and enrich them. Any organization can create an enormous amount of log events in a short period, so the creation of an efficient and dependable pipeline is crucial to maintaining the integrity and stability of any logging solution. The multitude of log formats and transport protocols are discussed, as well as how to decide on the best configuration for any given situation. Traditionally, log parsing has been painful and full of potential error, but the techniques shown throughout this course day will reduce or eliminate this pain and teach students how to substitute legacy solutions with more modern and efficient solutions. By the end of day 2, students will be familiar with optimal logging formats and with new and effective ways to parse legacy or difficult-to-handle formats.

While having perfectly parsed logs is great on its own, we can go much further. The value of a parsed log can be improved hundreds of times over with proper enrichment and with nominal performance impacts on log ingestion rates. Log enrichment includes adding context to logs and various other techniques used to increase your detection capabilities. Additionally, conditional logic and strategies for log filtering are discussed to ensure that the system will not be slowed by processing unneeded information.

The final piece of SIEM architecture is collecting logs off edge devices. Many organizations are unwilling or unable to deploy agent-based log collection, so both agent and agentless methods of log collection are discussed so that students can identify their ideal deployment. Although many students may already have a SIEM system in their environment, the Elastic set of tools can also be used to further supplement and improve the performance of other commercial SIEMs. We'll explain new trends such as the dual-stack SIEM environment, and examine how to use Logstash to supplement pre-existing SIEM deployments that struggle with high volume issues and poor data enrichment features. Alerting based on logs is also covered, with a review of both Elastic and third-party solutions.

Exercises

• Installing and Configuring Logstash - You'll learn to install and tune Logstash. Tuning includes memory optimizations, setting up X-Pack monitoring, and verifying if Logstash is set to run on reboot automatically.
• Traditional Parsing - You will use Logstash to parse syslog manually using both regex and patterns. Next, you'll apply UTC to the log's time field and add tags for context. Finally, you'll ship off logs into Elasticsearch and view them with Kibana.
• Modern Parsing - You'll apply automatic parsing of modern log formats such as key-value and JSON with purpose-built Logstash plugins.
• Filtering and Enrichment - This involves applying log filters to control the volume of data going into your SIEM. Also, you'll add context and field cleanup techniques via log enrichment to make analysis better.
• Log Agents - You'll install and use Filebeat and NXLog to collect logs and send them through the entire log pipeline. You'll learn how to use agents over various transport mechanisms such as TCP versus UDP.
• Installing and Configuring Elasticsearch - The first step in our SIEM involves installing Elasticsearch from scratch and configuring it to be production-ready and usable for the rest of the class.
• Cluster Creation and Management - You'll set up Cerebro for cluster and index management, creating an Elasticsearch cluster and using index templates for routing and understanding node failover.
• Kibana - You'll learn how to use the Kibana interface to run searches and create visualizations and dashboards, then use them to explore and answer questions about your data.
• Securing the Stack - You'll use X-Pack to enable Authentication, Authorization, and Auditing, as well as encryption between nodes and to Kibana.

CPE/CMU Credits: 7

Topics

• Log Aggregation
  • General Architecture
  • Open-source Solutions
  • Scaling Out (Load Balancing, Multiple Nodes, Docker)
  • Synchronizing Configurations across Multiple Nodes
  • Handling EPS
  • Performance Tuning
  • Input/Filter/Output
• Traditional Parsing
  • Syslog
  • Regex
  • Grok
  • Patterns
• Modern Parsing
  • Log Formats (CSV, KV, JSON, XML)
  • Automatic Parsers
  • Fixing Broken Log Formats
  • Extraction vs. Parsing
• Log Enrichment
  • Log Filtering Techniques
  • Field Standardization
  • Tags
  • Conditional Filters
  • Augmentation (geoip, DNS, etc.)
  • Debugging (Ingest Time)
  • Custom Enrichment (Ruby)
  • Performance Impact
• Agents and Log Collection
  • Architecture
  • Core Features
  • Automatic Configuration Techniques
  • Endpoint Filtering
  • Scaling
  • Automatic Configuration Control
  • Scripts
• Third-Party Integration and Dual-Stack SIEM
  • Compliance vs. Tactical
  • Commercial vs. Open-source
  • Duplicating Data to Multiple Sources
  • Converting Output Format per SIEM
  • Using Message Brokers
  • Migrating Into or Out of Elastic
• Alerting
  • Alert Engines and How They Function
  • Rule Types
  • Rule Development
  • Rule Testing

What You Will Receive

• Custom distribution of Linux with software ready to setup your own custom SIEM
• Realistic log data
• MP3 audio files of the complete course lecture
• Intro and Walkthrough videos of labs with advanced functionality such as text searching and navigation
• Digital wiki with labs
• USB 3.0 stick that includes the above and more