David McGinnisMar 35 minDebugging From The Field: When Parallelization Goes WrongBlog post chronicling the investigation of a series of errors in a Spark Streaming job reading from Spark.
David McGinnisFeb 257 minDebugging From The Field: The Case of the Empty FilesA team at a client was using Spark to read and write to a Kafka topic. [...] files that would be written that were completely empty.
David McGinnisDec 3, 20195 minDebugging From The Field: The Case of the Ignored Configuration ChangeWe made the change on a Sunday, but four days later, the number of files had not appreciably changed in the YARN logs directory.
David McGinnisNov 12, 20194 minDebugging from the Field: Smartsense Activity Explorer Stops WorkingThis client didn't use the activity explorer much [...] I tried to run one of the paragraphs and immediately [failed].
David McGinnisSep 16, 20194 minDebugging from the Field: Sudden En Masse Failures in 100s of Spark Streaming JobsAs this case shows, it is very important to understand where to look and how to filter out noise from the signal you care about.
David McGinnisSep 8, 20194 minDebugging From the Field: Sudden Kerberos Failure in HiveServer2 InstanceA client has a medium sized kerberized HDP 2.6.0 installation. Hive Interactive is disabled and Hive is set up to use HTTP transport mode.
