BLOG | Understanding Data.

David McGinnis

Feb 2, 20224 min read

Abstracting External Services with I/O Interfaces - Part 2

David McGinnis

Jan 26, 20225 min read

Abstracting External Services with I/O Interfaces - Part 1

While this is a concept I've used at many clients, I've never seen this concept approached elsewhere. Because of this, all of the...

David McGinnis

Jan 19, 20224 min read

Debugging from the Field: Sudden CI Test Failures

David McGinnis

Jan 12, 20222 min read

A New Start

David McGinnis

Jun 23, 20202 min read

Personal Update

David McGinnis

Mar 17, 20206 min read

Testing a Hive Patch on a Local System

David McGinnis

Mar 10, 20204 min read

A Crash Course in Proper Oozie Usage

David McGinnis

Mar 3, 20205 min read

Debugging From The Field: When Parallelization Goes Wrong

David McGinnis

Feb 25, 20207 min read

Debugging From The Field: The Case of the Empty Files

David McGinnis

Feb 18, 20204 min read

Spark Job Optimization Myth #6: I'm Seeing Out of Memory Exceptions, So I Need to Increase Memory

David McGinnis

Feb 11, 20205 min read

Spark Job Optimization Myth #5: Increasing Executor Cores is Always a Good Idea

David McGinnis

Feb 4, 20205 min read

Spark Job Optimization Myth #4: I Need More Overhead Memory

David McGinnis

Jan 28, 20205 min read

Spark Job Optimization Myth #3: I Need More Driver Memory

David McGinnis

Jan 21, 20205 min read

Spark Job Optimization Myth #2: Increasing the Number of Executors Always Improves Performance

David McGinnis

Jan 6, 20205 min read

Spark Job Optimization Myth #1: Increasing the Memory Per Executor Always Improves Performance

David McGinnis

Dec 3, 20195 min read

Debugging From The Field: The Case of the Ignored Configuration Change

David McGinnis

Nov 26, 20194 min read

A Crash Course in YARN Log Aggregation

David McGinnis

Nov 20, 20195 min read

Tracking Down Small Files with Smartsense Dashboards

David McGinnis

Nov 12, 20194 min read

Debugging from the Field: Smartsense Activity Explorer Stops Working

David McGinnis

Nov 5, 20196 min read

Abstracting External Services with I/O Interfaces - Part 2

Abstracting External Services with I/O Interfaces - Part 1

Debugging from the Field: Sudden CI Test Failures

A New Start

Personal Update

Testing a Hive Patch on a Local System

A Crash Course in Proper Oozie Usage

Debugging From The Field: When Parallelization Goes Wrong

Debugging From The Field: The Case of the Empty Files

Spark Job Optimization Myth #6: I'm Seeing Out of Memory Exceptions, So I Need to Increase Memory

Spark Job Optimization Myth #5: Increasing Executor Cores is Always a Good Idea

Spark Job Optimization Myth #4: I Need More Overhead Memory

Spark Job Optimization Myth #3: I Need More Driver Memory

Spark Job Optimization Myth #2: Increasing the Number of Executors Always Improves Performance

Spark Job Optimization Myth #1: Increasing the Memory Per Executor Always Improves Performance

Debugging From The Field: The Case of the Ignored Configuration Change

A Crash Course in YARN Log Aggregation

Tracking Down Small Files with Smartsense Dashboards

Debugging from the Field: Smartsense Activity Explorer Stops Working

Spark Job Optimization: Dealing with Data Skew

SIGN UP AND STAY UPDATED!