Issue Report

All Posts
Administration
Machine Learning
Security
Development
Issue Report
Tutorial

Debugging From The Field: When Parallelization Goes Wrong

David McGinnis

Mar 3
5 min

Debugging From The Field: When Parallelization Goes Wrong

Blog post chronicling the investigation of a series of errors in a Spark Streaming job reading from Spark.

147 viewsWrite a comment

Debugging From The Field: The Case of the Empty Files

David McGinnis

Feb 25
7 min

Debugging From The Field: The Case of the Empty Files

A team at a client was using Spark to read and write to a Kafka topic. [...] files that would be written that were completely empty.

46 views1 comment

Debugging From The Field: The Case of the Ignored Configuration Change

David McGinnis

Dec 3, 2019
5 min

Debugging From The Field: The Case of the Ignored Configuration Change

We made the change on a Sunday, but four days later, the number of files had not appreciably changed in the YARN logs directory.

40 viewsWrite a comment

Debugging from the Field: Smartsense Activity Explorer Stops Working

David McGinnis

Nov 12, 2019
4 min

Debugging from the Field: Smartsense Activity Explorer Stops Working

This client didn't use the activity explorer much [...] I tried to run one of the paragraphs and immediately [failed].

131 viewsWrite a comment

Debugging from the Field: Sudden En Masse Failures in 100s of Spark Streaming Jobs

David McGinnis

Sep 16, 2019
4 min

Debugging from the Field: Sudden En Masse Failures in 100s of Spark Streaming Jobs

As this case shows, it is very important to understand where to look and how to filter out noise from the signal you care about.

22 viewsWrite a comment

Debugging From the Field: Sudden Kerberos Failure in HiveServer2 Instance

David McGinnis

Sep 8, 2019
4 min

Debugging From the Field: Sudden Kerberos Failure in HiveServer2 Instance

A client has a medium sized kerberized HDP 2.6.0 installation. Hive Interactive is disabled and Hive is set up to use HTTP transport mode.

114 viewsWrite a comment

{"items":["5e372808d8a25a0018c6eeaa","5e2cb56bf75427001767e828","5ddaed4bce4ab700173949dc","5dc3484533b95000170484d8","5d6ea994c3a3fe00c965e50f","5d65fb4c7da1c501f185483c"],"styles":{"galleryType":"Columns","groupSize":1,"showArrows":true,"cubeImages":true,"cubeType":"fill","cubeRatio":1.3333333333333333,"isVertical":true,"gallerySize":30,"collageAmount":0,"collageDensity":0,"groupTypes":"1","oneRow":false,"imageMargin":32,"galleryMargin":0,"scatter":0,"chooseBestGroup":true,"smartCrop":false,"hasThumbnails":false,"enableScroll":true,"isGrid":true,"isSlider":false,"isColumns":false,"isSlideshow":false,"cropOnlyFill":false,"fixedColumns":1,"enableInfiniteScroll":true,"isRTL":false,"minItemSize":50,"rotatingGroupTypes":"","rotatingCubeRatio":"","gallerySliderImageRatio":1.7777777777777777,"numberOfImagesPerRow":1,"numberOfImagesPerCol":1,"groupsPerStrip":0,"borderRadius":0,"boxShadow":0,"gridStyle":1,"mobilePanorama":false,"placeGroupsLtr":false,"viewMode":"preview","thumbnailSpacings":4,"galleryThumbnailsAlignment":"bottom","isMasonry":false,"isAutoSlideshow":false,"slideshowLoop":false,"autoSlideshowInterval":4,"bottomInfoHeight":0,"titlePlacement":"SHOW_ON_THE_RIGHT","galleryTextAlign":"center","scrollSnap":false,"itemClick":"nothing","fullscreen":true,"videoPlay":"hover","scrollAnimation":"NO_EFFECT","slideAnimation":"SCROLL","scrollDirection":0,"overlayAnimation":"FADE_IN","arrowsPosition":0,"arrowsSize":23,"watermarkOpacity":40,"watermarkSize":40,"useWatermark":true,"watermarkDock":{"top":"auto","left":"auto","right":0,"bottom":0,"transform":"translate3d(0,0,0)"},"loadMoreAmount":"all","defaultShowInfoExpand":1,"allowLinkExpand":true,"expandInfoPosition":0,"allowFullscreenExpand":true,"fullscreenLoop":false,"galleryAlignExpand":"left","addToCartBorderWidth":1,"addToCartButtonText":"","slideshowInfoSize":200,"playButtonForAutoSlideShow":false,"allowSlideshowCounter":false,"hoveringBehaviour":"NEVER_SHOW","thumbnailSize":120,"magicLayoutSeed":1,"imageHoverAnimation":"NO_EFFECT","imagePlacementAnimation":"NO_EFFECT","calculateTextBoxWidthMode":"PERCENT","textBoxHeight":0,"textBoxWidth":200,"textBoxWidthPercent":50,"textImageSpace":10,"textBoxBorderRadius":0,"textBoxBorderWidth":0,"loadMoreButtonText":"","loadMoreButtonBorderWidth":1,"loadMoreButtonBorderRadius":0,"imageInfoType":"ATTACHED_BACKGROUND","itemBorderWidth":1,"itemBorderRadius":0,"itemEnableShadow":false,"itemShadowBlur":20,"itemShadowDirection":135,"itemShadowSize":10,"imageLoadingMode":"BLUR","expandAnimation":"NO_EFFECT","imageQuality":90,"usmToggle":false,"usm_a":0,"usm_r":0,"usm_t":0,"videoSound":false,"videoSpeed":"1","videoLoop":true,"gallerySizeType":"px","gallerySizePx":940,"allowTitle":true,"allowContextMenu":true,"textsHorizontalPadding":-30,"itemBorderColor":{"value":"rgb(179, 179, 179)"},"showVideoPlayButton":true,"galleryLayout":2,"targetItemSize":940,"selectedLayout":"2|bottom|1|fill|true|0|true","layoutsVersion":2,"selectedLayoutV2":2,"isSlideshowFont":false,"externalInfoHeight":0,"externalInfoWidth":0.5},"container":{"width":940,"galleryWidth":972,"galleryHeight":0,"scrollBase":0,"height":null}}

SIGN UP AND STAY UPDATED!