Skip to content
All projects
Data Analytics

Job Market Analytics

A Pandas pipeline and SQL aggregation over employment datasets, ending in Power BI dashboards — 10,000 job postings across 39 columns cleaned to zero missing values and zero duplicates, with the delimited skills field unpivoted for frequency analysis.

Key result

10K postings— Cleaned to zero nulls, shipped as dashboards

01Problem

Raw job-posting data arrives messy: inconsistent titles, missing salaries, duplicate listings, and required skills packed into a single delimited free-text field that cannot be counted as-is.

02Objective

Build a repeatable pipeline rather than a one-off notebook — clean and aggregate the postings, then publish the result as Power BI dashboards a non-technical stakeholder can explore.

03Architecture

  1. Employment Datasets
  2. Pandas Pipeline
  3. SQL Aggregation
  4. Power BI
  5. Dashboards
Pipeline from raw employment data through to dashboards

04Technology

  • Python
  • Pandas
  • SQL
  • Power BI
  • Matplotlib

05Implementation

Key technical decisions

  1. 01Profiled the raw data before transforming anything, so cleaning decisions were driven by what was actually wrong with it.
  2. 02Unpivoted the delimited skills field into a long table, because skill frequency cannot be counted while skills live inside one string.
  3. 03Reviewed salary outliers with the IQR method rather than deleting them automatically.
  4. 04Handed the aggregation to SQL and kept Pandas for cleaning and reshaping.

What was built

  • Pandas pipeline handling type coercion, null treatment, deduplication, and salary normalization.
  • Skills column unpivoted to roughly 112,000 skill-posting rows for frequency analysis.
  • SQL aggregation over the cleaned tables.
  • Matplotlib charts across distribution and segment comparison.
  • Power BI dashboards over the cleaned model, with KPI cards, charts, and slicers.

06Data

Source dataset and the modeled output

  • 10,000 job postings across 39 columns
  • Role title, company, location, employment type, and remote status
  • Salary ranges arriving in mixed formats
  • Required skills stored as delimited free text, unpivoted to ~112K skill rows

07Results

0

Missing values after cleaning

0

Duplicate records remaining

112K

Skill rows after unpivot

Power BI

Dashboards over the cleaned model

10Links

GitHub