Objective
Pulling a unique list of source IPs out of a raw IIS log is the kind of task that looks trivial and then quietly eats an hour. The log is space-delimited, the client IP sits in the middle of a long line, and the same address repeats across hundreds of entries:
2025-06-10 12:00:01 192.168.1.10 GET /index.html - 80 - 66.249.66.1 Mozilla/5.0+(Windows+NT+10.0;...) 200 0 0
2025-06-10 12:00:03 192.168.1.11 GET /products/view?id=123 - 80 - 172.217.9.46 Mozilla/5.0+(Windows+NT+10.0;...) 200 0 0
2025-06-10 12:00:05 192.168.1.12 GET /about-us.html - 80 - 66.102.7.1 Mozilla/5.0+(Windows+NT+10.0;...) 200 0 0Doing this by hand is slow, and worse, it’s error-prone. The objective was to automate the whole pipeline: extract the IPs, deduplicate them, and print or forward them for action.
Setup
Nothing beyond a Python interpreter and the log file itself. The file path lives in an environment variable rather than being hard-coded, so the same script travels between hosts without edits.
Implementation
Armed with some Python knowledge from my bachelor’s program, here’s the clean, reusable version of the script:
import re
import os
# Access the environment variable containing the log file path
file_location = os.getenv("LinkedIN_Var")
# Regular expression for matching IPv4 Addresses
ipv4_pattern = r'\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b'
# Read the log file contents
with open(file_location, 'r') as iis_input:
log_contents = iis_input.read()
# Extract and deduplicate IP addresses
matches = re.findall(ipv4_pattern, log_contents)
seen_ips = set()
unique_ips = [ip for ip in matches if not (ip in seen_ips or seen_ips.add(ip))]
# Output could be redirected, stored, or sent via API
print(unique_ips)Why the one-liner, not a set The deduplication one-liner preserves first-seen order rather than sorting, which matters when you’re trying to read the sequence of a scan.
set()alone would throw that ordering away.
Findings
The script reduced analysis time from an hour to a couple of seconds. Better yet, it eliminated human error and laid the groundwork for scaling into a real-time automated workflow, piping IPs directly into a SIEM or SOAR.
The pattern generalizes. Today I use similar scripts across a range of workflows:
- Parsing unstructured data
- Normalizing threat intel
- Preprocessing logs for enrichment and correlation
That last one grew directly into the Threat-Enriched Log Pipeline.
Next Steps
If you’re new to security operations, the lesson here is worth more than the script: look for opportunities to reduce friction.
Whether it’s a Python one-liner, a shell script, or a regex expression, these small optimizations compound into massive operational gains. Don’t just learn to read logs. Learn to control them.