Storage drives the workflow
Blob triggers connect incoming files to chunk creation, then chunk creation to processing.
Cloud / Event-driven systems
Distributed Log Processing
From noisy logs
to queryable signals.
SYSTEM / LOG-PROCESSING01 → 04
01Incoming blobUploaded log file
02SplitterCreate smaller chunks
03ProcessorParse + identify errors
04Azure SQLPersist ErrorLogs
2Azure Functions
2blob containers
SQLstructured error records
An event-driven Azure pipeline that splits uploaded log files, extracts errors from each chunk, and stores them for querying.
Make large logs manageable.
Large log files are difficult to inspect as a single unit. Error records need to be separated from the raw text and stored in a form that can be queried.
A deliberate path through the problem.
Uploading a file to the incoming container triggers a splitter. Each resulting chunk triggers a processor that parses log entries and inserts identified errors into Azure SQL.
PythonAzure FunctionsBlob StorageAzure SQLpyodbc
The choices that shape the work.
Two focused functions
Splitting and parsing live in separate function directories so each step has a clear job.
A structured destination
The documented ErrorLogs schema stores a log identifier, original text, category, and creation time.
What the project delivers.
The repository provides both functions, sample log files, a database schema, and Azure setup instructions for recreating the pipeline.
Let events carry the work.
A storage event becomes a useful boundary: each stage works on a smaller artifact and hands the next stage a durable input.