Load data that has no API into Snowflake, Databricks, or S3
Stepwork pulls reports from admin consoles, ERPs, and vendor portals that have no API or connector, on a schedule, and loads the rows into Snowflake, Databricks, or S3.
- Use case
- Scheduled extraction from UI-only sources into the data warehouseIT
- Best for
- Data Engineer, Analytics Lead, IT Systems Manager
- Applications used
Snowflake
Databricks
Amazon S3
NetSuiteOkta
- Business outcome
- Fresh tables from sources no connector reaches
The problem
No API, no connector. Just an export button.
- Sign in to each admin console and vendor portal
- Set the filters and date range on every report
- Export the CSV, or copy the table page by page
- Fix the columns that moved since last month
- Upload the file to the warehouse or a shared drive
- Find out a month later that a pull was missed
Systems
Pulls from the tools your connectors skip
Managed connectors cover the systems with an API. Stepwork reads the rest through their own screens, with a login your team grants, and hands the rows to the warehouse or bucket your pipelines already read.
- Warehouses and storage: Snowflake, Databricks, Amazon S3, Google BigQuery, Amazon Redshift
- ERP and finance: NetSuite, SAP Business One, Sage 100, Microsoft Dynamics GP, Coupa, Ramp
- Identity and SaaS admin consoles: Okta, Microsoft Entra ID, Slack, Salesforce, GitHub, Zoom
- HR and payroll: Workday, ADP Workforce Now, BambooHR, Greenhouse
- Security and cloud consoles: CrowdStrike, Wiz, Tenable, AWS, Datadog
Example procedures
Two pulls, step by step
One repeats for every app on your list and checks the columns before it loads. One checks that an ERP report adds up before it lands. Swap in your warehouse above each one and the steps stay the same.
Weekly user and activity snapshot from every admin console into Snowflake
Excel
Okta
Salesforce
GitHub
Snowflake
Slack
Step by step
- The app list comes inStepwork reads the snapshot list your data team keeps in Excel: each app, the table its rows go to, and the columns that table expects. Each row is one pass through the loop.
- Each admin console gets readInside the solid frame, Stepwork runs the extraction recorded for that app's admin console and reads every user, role, and last sign-in, page by page.Repeats: For each app on the list · 24 apps
- The rule checks the shapeIf the columns don't match the ones the table expects, the run takes the top path. If they match, the rows go on to load.
- If the columns changedFlag the change in Slack
- If they matchWrite the rows to Snowflake
Repeats: For each app on the list · 24 apps - Changes get flaggedOnly when the columns changed: the data team gets a message in Slack naming the app and the columns that moved, and nothing loads for that app.Only on this path: If the columns changedRepeats: For each app on the list · 24 apps
- The rows get writtenThe API card writes the rows to the app's table in Snowflake.Only on this path: If they matchRepeats: For each app on the list · 24 apps
- The data team gets the pictureAfter the last app, the Summary card lists how many rows each table got and which apps need a look.
Field map
| From | To |
|---|---|
Month-end AR aging from NetSuite into Snowflake
NetSuiteSnowflake
Slack
Step by step
- The report runsStepwork signs in to NetSuite, runs the AR aging saved search for the close, and reads every row across every page, plus the report's total line.
- The rule checks the totalIf the rows don't add up to the report's total, the run takes the top path. If they tie out, they go on to load.
- If they don't add upSend the gap to finance in Slack
- If they tie outWrite the rows to Snowflake
- Gaps go to financeOnly when they don't add up: finance gets the report total and what was read in Slack, and nothing half-read lands in Snowflake.Only on this path: If they don't add up
- The rows get writtenThe API card writes the aging rows to the ar_aging table in Snowflake.Only on this path: If they tie out
- Finance hears it's currentThe Summary card tells finance the table is current, with the row count and the total it tied out to.Only on this path: If they tie out
Field map
| From | To |
|---|---|
Flows
Extraction flows these procedures run
Each flow is one recorded pull from one system's screens. Procedures run them on a schedule and hand the rows to your warehouse.
- Okta
Extract full user directory with status
Pages through the Okta user list and extracts name, email, status, and last login for every account into a table. - Slack
Extract member list with last activity
Pages through Slack member management and extracts name, email, role, and last-active date for every member. - Salesforce
Extract active users with license types
Extracts active users with license types, profiles, and last login into a table.
NetSuiteExtract AR aging and overdue invoices
Runs AR aging in NetSuite and extracts overdue invoices with customer, amount, and days past due for collections.- Workday
Extract headcount and open position snapshot
Runs the Workday headcount report and extracts filled positions, open requisitions, and pending hires by cost center. - AWS
Extract monthly cost by service
Opens Cost Explorer, sets the monthly view, and extracts spend by service into a table for reporting. - CrowdStrike
Extract detections summary
Signs into the Falcon console and extracts recent detections with severity, host, and status. - Zylo
Extract SaaS renewal calendar
Signs into Zylo and extracts contracts renewing in the next 90 days with vendor, value, owner, and auto-renew flag. - LinkedIn Campaign Manager
Export a demographic breakdown
Opens a campaign's demographics report for the period and exports the company and job-title rows.
More procedures
More reporting procedures
Ready-made procedures in the Marketplace that pull from several tools at once.
2 more
Google Workspace · Microsoft 365 · Atlassian + 2 moreMonthly license utilization report
Extracts license usage from Google Workspace, Microsoft 365, Atlassian, Zoom, and Adobe on the first of the month, consolidates seats-owned versus seats-active per platform, and posts the utilization table with waste estimates to the IT leadership channel.1 more
AWS · Microsoft Azure · Google Cloud Platform + 1 moreCloud cost weekly digest
Extracts weekly cost by service from AWS, cost by resource group from Azure, and compute utilization from GCP, flags idle resources and week-over-week jumps, and posts the consolidated digest to the platform channel.- Looker · Tableau · Slack
Monday leadership reporting pack
Every Monday, downloads the dashboard snapshots from Looker and workbook exports from Tableau that make up the leadership pack, assembles them in order, and delivers the pack to the leadership channel and email list before the staff meeting.
Why Stepwork
The same pull, the same columns, every run
Stepwork runs the extraction you recorded once, so every run opens the same report with the same filters and hands over the same columns. A rule checks the shape before anything loads, so a changed report raises a flag instead of breaking a table. Every load traces back to a run record: which report, which login, how many rows, and when.
FAQ
Common questions
The math
What the manual version costs a data team
Example volumes for a team pulling 25 reports a week from tools without a connector. Swap in yours.
| The manual task | Times a week | Minutes each | Hours a week |
|---|---|---|---|
| Export a report from an admin console or portal | 25 | 10 | 4.2 |
| Clean and reshape the export | 25 | 15 | 6.3 |
| Upload and load the file | 25 | 5 | 2.1 |
| Redo a pull that broke or was missed | 3 | 30 | 1.5 |
- Hours a week
- 14
- Hours a year
- 700
- Of one full-time person
- 35%
