Skip to content

Common Workflows

Learn efficient workflows for using the SQLStream interactive shell.

Data Exploration Workflow

Step-by-Step

  1. Launch shell with file: 

    sqlstream shell data.csv

  2. Check schema (F2 to toggle schema browser):

  3. See all columns and types

  4. Check row counts

  5. Run exploratory queries in separate tabs:

  6. Tab 1: SELECT * FROM data LIMIT 10
  7. Tab 2: SELECT COUNT(*) FROM data

  8. Tab 3: SELECT DISTINCT category FROM data

  9. Use query history (Ctrl+Up/Down) to refine queries

  10. Export results when satisfied (Ctrl+X)

Multi-File Analysis Workflow

Joining Multiple Files

  1. Launch shell without file: 

    sqlstream shell

  2. Open file browser (Ctrl+O)

  3. Navigate and load first file

  4. Repeat for additional files

  5. Check schemas (F2) for all loaded files

  6. Write JOIN query: 

    SELECT *
FROM 'customers.csv' c
JOIN 'orders.csv' o ON c.id = o.customer_id
WHERE o.amount > 1000

  7. Toggle to DuckDB backend (F5) for better JOIN performance

  8. Filter and export results

Report Generation Workflow

Creating Analysis Reports

  1. Tab 1: Summary statistics 

    SELECT
    COUNT(*) as total_records,
    SUM(amount) as total_revenue,
    AVG(amount) as avg_order_value
FROM sales

  2. Tab 2: Breakdown by category 

    SELECT category, COUNT(*) as count, SUM(amount) as revenue
FROM sales
GROUP BY category
ORDER BY revenue DESC

  3. Tab 3: Top performers 

    SELECT name, SUM(amount) as total
FROM sales
GROUP BY name
ORDER BY total DESC
LIMIT 10

  4. Export each result to different formats (CSV, JSON)

  5. State persists - reopen shell later to continue work

Data Quality Checking Workflow

Validating Data

  1. Check for nulls: 

    SELECT COUNT(*) FROM data WHERE important_field IS NULL

  2. Check for duplicates: 

    SELECT id, COUNT(*) as count
FROM data
GROUP BY id
HAVING count > 1

  3. Validate ranges: 

    SELECT * FROM data
WHERE age < 0 OR age > 120

  4. Use filter (Ctrl+F) to explore problematic records

  5. Export issues for further investigation

Performance Optimization Workflow

Optimizing Slow Queries

  1. Run query with Python backend (default)

  2. Check execution time in status bar

  3. View query plan (F4) to understand execution

  4. Try Pandas backend (F5):

  5. Good for large files, simple queries

  6. Try DuckDB backend (F5 again):

  7. Best for complex SQL

  8. Compare times and choose best backend

  9. Refine query based on plan:

  10. Add WHERE filters early
  11. Select only needed columns
  12. Use appropriate indexes

Iterative Development Workflow

Developing Complex Queries

  1. Tab 1: Start simple 

    SELECT * FROM data LIMIT 10

  2. Refine incrementally using history (Ctrl+Up):

  3. Add WHERE clause
  4. Add GROUP BY

  5. Add ORDER BY

  6. Tab 2: Test subquery separately 

    SELECT category, AVG(amount) as avg_amount
FROM data
GROUP BY category

  7. Tab 3: Combine into final query 

    WITH category_avg AS (
    SELECT category, AVG(amount) as avg_amount
    FROM data
    GROUP BY category
)
SELECT d.*, c.avg_amount
FROM data d
JOIN category_avg c ON d.category = c.category
WHERE d.amount > c.avg_amount

  8. Save state (Ctrl+S) at checkpoints

Tips for Efficient Workflows

Speed Up Your Work

  • Use tabs for context: Keep different analyses in separate tabs
  • Leverage history: Press Ctrl+Up instead of retyping queries
  • Toggle layout (Ctrl+L): More editor space for complex queries
  • File browser (Ctrl+O): Quickly switch between data files
  • Filter results (Ctrl+F): Narrow down without re-running query
  • Auto-save: Your work persists between sessions

Keyboard-First Navigation

  • Minimize mouse usage with keyboard shortcuts
  • F2/F3 for sidebars, F4 for plans, F5 for backends
  • Ctrl+Tab to switch tabs quickly
  • Ctrl+Enter to execute without reaching for mouse

See Also