Date of Award

Summer 2024

Document Type

Open Access Dissertation

Department

Computer Science and Engineering

First Advisor

Jorge Crichigno

Abstract

Network security has become increasingly essential in today's networks due to the growth of various network applications, such as Machine Learning (ML) and Fifth-Generation (5G) Networks. One fundamental Internet protocol is the Domain Name System (DNS), which maps domain names to Internet Protocol (IP) addresses. Despite its importance, DNS traffic is often forwarded without being analyzed, making it a center of ever-evolving attacks. Traditionally, defense strategies are implemented on fixed-function security middleboxes that are costly, proprietary, and hard to manage. Alternatively, defenses implemented in software use general-purpose servers (e.g., the control plane of a Software-Defined Networking (SDN) network) that cannot keep up with high-speed traffic rates. Additionally, detecting malicious behavior in the presence of encryption protocols often requires collecting several network traffic artifacts, which is a difficult task at scale. Recently, Programmable Data Plane (PDP) switches have attracted significant attention from the research community and industry, permitting operators to develop and run customized packet processing functions. PDP Switches rely on domain-specific processors, capable of processing packets at terabits per second (Tbps) rates and running customized applications. This dissertation leverages PDP switches to address the aforementioned weak points and provide enhanced security at high-speed traffic rates. The contributions are: (i) Parsing and analyzing DNS domain names solely in the data plane. In particular, Deep Packet Inspection (DPI) is leveraged to extract the domain name of any number of subdomains (labels), thus enforcing security policies (e.g., blocking malicious domains). The system presents significant performance gains compared to a software-based firewall. (ii) Leveraging the PDP to classify malware families that exploit DNS to evade detection. The PDP switch is harnessed to extract a combination of unique network heuristics and domain name features through DPI. These features are sent to the control plane to classify Domain Generation Algorithm (DGA)-based malware. (iii) Developing a system that performs an in-network DGA detection via an ML module implemented in the PDP switch at line rate. The control plane is activated for enhanced detection and classification of DGAs. The framework immediately halts malicious communications and shows promising preliminary results on encrypted DNS traffic. (iv) Inspecting services and performing fine-grained and coarse-grained monitoring of HyperText Transfer Protocol Secure (HTTPS) traffic using PDPs. The PDP switch performs DPI on the Transport Layer Security (TLS) protocol at line rate. (v) Performing a STRIDE model assessment of main PDP applications, discussing plausible remediation solutions, and finally, identifying PDP's security challenges and trends.

Rights

© 2024, Ali AlSabeh

Download

Share

COinS