[June 12, 2025 at 9:00:58 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1535115264
[June 12, 2025 at 9:00:58 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1535115264
[June 12, 2025 at 9:00:58 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1535115264
21:00:58.745 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
21:00:58.745 INFO CollectMultipleMetricsSpark - The Genome Analysis Toolkit (GATK) v4.6.2.0-8-g092bdf2-SNAPSHOT
21:00:58.745 INFO CollectMultipleMetricsSpark - For support and documentation go to https://software.broadinstitute.org/gatk/
21:00:58.745 INFO CollectMultipleMetricsSpark - Executing as root@e24cea241624 on Linux v6.11.0-1015-azure amd64
21:00:58.745 INFO CollectMultipleMetricsSpark - Java runtime: OpenJDK 64-Bit Server VM v17.0.12+7-Ubuntu-1ubuntu222.04
21:00:58.745 INFO CollectMultipleMetricsSpark - Start Date/Time: June 12, 2025 at 9:00:58 PM GMT
21:00:58.745 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
21:00:58.745 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
21:00:58.745 INFO CollectMultipleMetricsSpark - HTSJDK Version: 4.2.0
21:00:58.745 INFO CollectMultipleMetricsSpark - Picard Version: 3.4.0
21:00:58.745 INFO CollectMultipleMetricsSpark - Built for Spark Version: 3.5.0
21:00:58.745 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.COMPRESSION_LEVEL : 2
21:00:58.745 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
21:00:58.745 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
21:00:58.745 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
21:00:58.745 INFO CollectMultipleMetricsSpark - Deflater: IntelDeflater
21:00:58.745 INFO CollectMultipleMetricsSpark - Inflater: IntelInflater
21:00:58.745 INFO CollectMultipleMetricsSpark - GCS max retries/reopens: 20
21:00:58.745 INFO CollectMultipleMetricsSpark - Requester pays: disabled
21:00:58.745 WARN CollectMultipleMetricsSpark -
?[1m?[31m !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Warning: CollectMultipleMetricsSpark is a BETA tool and is not yet ready for use in production
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!?[0m
21:00:58.745 INFO CollectMultipleMetricsSpark - Initializing engine
21:00:58.745 INFO CollectMultipleMetricsSpark - Done initializing engine
21:00:58.745 INFO CollectMultipleMetricsSpark - Spark verbosity set to INFO (see --spark-verbosity argument)
WARNING 2025-06-12 21:00:58 SamReaderFactory Unable to detect file format from input URL or stream, assuming SAM format.
WARNING 2025-06-12 21:00:58 SamReaderFactory Unable to detect file format from input URL or stream, assuming SAM format.
21:00:58.747 INFO MemoryStore - Block broadcast_3356 stored as values in memory (estimated size 37.8 KiB, free 1913.6 MiB)
21:00:58.747 INFO MemoryStore - Block broadcast_3356_piece0 stored as bytes in memory (estimated size 8.4 KiB, free 1913.6 MiB)
21:00:58.747 INFO BlockManagerInfo - Added broadcast_3356_piece0 in memory on localhost:43253 (size: 8.4 KiB, free: 1918.3 MiB)
21:00:58.747 INFO SparkContext - Created broadcast 3356 from broadcast at SamSource.java:78
21:00:58.748 INFO MemoryStore - Block broadcast_3357 stored as values in memory (estimated size 305.5 KiB, free 1913.3 MiB)
21:00:58.755 INFO MemoryStore - Block broadcast_3357_piece0 stored as bytes in memory (estimated size 64.1 KiB, free 1913.2 MiB)
21:00:58.755 INFO BlockManagerInfo - Added broadcast_3357_piece0 in memory on localhost:43253 (size: 64.1 KiB, free: 1918.3 MiB)
21:00:58.755 INFO SparkContext - Created broadcast 3357 from newAPIHadoopFile at SamSource.java:108
WARNING 2025-06-12 21:00:58 SamReaderFactory Unable to detect file format from input URL or stream, assuming SAM format.
WARNING 2025-06-12 21:00:58 SamReaderFactory Unable to detect file format from input URL or stream, assuming SAM format.
21:00:58.757 INFO MemoryStore - Block broadcast_3358 stored as values in memory (estimated size 37.8 KiB, free 1913.2 MiB)
21:00:58.757 INFO MemoryStore - Block broadcast_3358_piece0 stored as bytes in memory (estimated size 8.4 KiB, free 1913.2 MiB)
21:00:58.758 INFO BlockManagerInfo - Added broadcast_3358_piece0 in memory on localhost:43253 (size: 8.4 KiB, free: 1918.3 MiB)
21:00:58.758 INFO SparkContext - Created broadcast 3358 from broadcast at SamSource.java:78
21:00:58.758 INFO MemoryStore - Block broadcast_3359 stored as values in memory (estimated size 305.5 KiB, free 1912.9 MiB)
21:00:58.765 INFO MemoryStore - Block broadcast_3359_piece0 stored as bytes in memory (estimated size 64.1 KiB, free 1912.8 MiB)
21:00:58.765 INFO BlockManagerInfo - Added broadcast_3359_piece0 in memory on localhost:43253 (size: 64.1 KiB, free: 1918.2 MiB)
21:00:58.765 INFO SparkContext - Created broadcast 3359 from newAPIHadoopFile at SamSource.java:108
21:00:58.767 INFO FileInputFormat - Total input files to process : 1
21:00:58.770 INFO SparkContext - Starting job: count at CollectMultipleMetricsSparkIntegrationTest.java:130
21:00:58.771 INFO DAGScheduler - Got job 876 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) with 1 output partitions
21:00:58.771 INFO DAGScheduler - Final stage: ResultStage 2462 (count at CollectMultipleMetricsSparkIntegrationTest.java:130)
21:00:58.771 INFO DAGScheduler - Parents of final stage: List()
21:00:58.771 INFO DAGScheduler - Missing parents: List()
21:00:58.771 INFO DAGScheduler - Submitting ResultStage 2462 (MapPartitionsRDD[10129] at filter at CollectMultipleMetricsSpark.java:193), which has no missing parents
21:00:58.771 INFO MemoryStore - Block broadcast_3360 stored as values in memory (estimated size 34.8 KiB, free 1912.8 MiB)
21:00:58.772 INFO MemoryStore - Block broadcast_3360_piece0 stored as bytes in memory (estimated size 14.6 KiB, free 1912.8 MiB)
21:00:58.772 INFO BlockManagerInfo - Added broadcast_3360_piece0 in memory on localhost:43253 (size: 14.6 KiB, free: 1918.2 MiB)
21:00:58.772 INFO SparkContext - Created broadcast 3360 from broadcast at DAGScheduler.scala:1580
21:00:58.772 INFO DAGScheduler - Submitting 1 missing tasks from ResultStage 2462 (MapPartitionsRDD[10129] at filter at CollectMultipleMetricsSpark.java:193) (first 15 tasks are for partitions Vector(0))
21:00:58.772 INFO TaskSchedulerImpl - Adding task set 2462.0 with 1 tasks resource profile 0
21:00:58.772 INFO TaskSetManager - Starting task 0.0 in stage 2462.0 (TID 1881) (localhost, executor driver, partition 0, PROCESS_LOCAL, 9688 bytes)
21:00:58.773 INFO Executor - Running task 0.0 in stage 2462.0 (TID 1881)
21:00:58.774 INFO NewHadoopRDD - Input split: file:/gatkCloneMountPoint/src/test/resources/org/broadinstitute/hellbender/metrics/analysis/CollectInsertSizeMetrics/insert_size_metrics_test.sam:0+44008
21:00:58.775 INFO Executor - Finished task 0.0 in stage 2462.0 (TID 1881). 946 bytes result sent to driver
21:00:58.775 INFO TaskSetManager - Finished task 0.0 in stage 2462.0 (TID 1881) in 3 ms on localhost (executor driver) (1/1)
21:00:58.775 INFO TaskSchedulerImpl - Removed TaskSet 2462.0, whose tasks have all completed, from pool
21:00:58.775 INFO DAGScheduler - ResultStage 2462 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) finished in 0.004 s
21:00:58.775 INFO DAGScheduler - Job 876 is finished. Cancelling potential speculative or zombie tasks for this job
21:00:58.775 INFO TaskSchedulerImpl - Killing all running tasks in stage 2462: Stage finished
21:00:58.775 INFO DAGScheduler - Job 876 finished: count at CollectMultipleMetricsSparkIntegrationTest.java:130, took 0.004856 s
21:00:58.775 INFO CollectMultipleMetricsSpark - Shutting down engine
[June 12, 2025 at 9:00:58 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1535115264
21:00:58.790 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
21:00:58.790 INFO CollectMultipleMetricsSpark - The Genome Analysis Toolkit (GATK) v4.6.2.0-8-g092bdf2-SNAPSHOT
21:00:58.790 INFO CollectMultipleMetricsSpark - For support and documentation go to https://software.broadinstitute.org/gatk/
21:00:58.790 INFO CollectMultipleMetricsSpark - Executing as root@e24cea241624 on Linux v6.11.0-1015-azure amd64
21:00:58.790 INFO CollectMultipleMetricsSpark - Java runtime: OpenJDK 64-Bit Server VM v17.0.12+7-Ubuntu-1ubuntu222.04
21:00:58.790 INFO CollectMultipleMetricsSpark - Start Date/Time: June 12, 2025 at 9:00:58 PM GMT
21:00:58.790 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
21:00:58.790 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
21:00:58.790 INFO CollectMultipleMetricsSpark - HTSJDK Version: 4.2.0
21:00:58.790 INFO CollectMultipleMetricsSpark - Picard Version: 3.4.0
21:00:58.790 INFO CollectMultipleMetricsSpark - Built for Spark Version: 3.5.0
21:00:58.790 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.COMPRESSION_LEVEL : 2
21:00:58.791 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
21:00:58.791 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
21:00:58.791 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
21:00:58.791 INFO CollectMultipleMetricsSpark - Deflater: IntelDeflater
21:00:58.791 INFO CollectMultipleMetricsSpark - Inflater: IntelInflater
21:00:58.791 INFO CollectMultipleMetricsSpark - GCS max retries/reopens: 20
21:00:58.791 INFO CollectMultipleMetricsSpark - Requester pays: disabled
21:00:58.791 WARN CollectMultipleMetricsSpark -
?[1m?[31m !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Warning: CollectMultipleMetricsSpark is a BETA tool and is not yet ready for use in production
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!?[0m
21:00:58.791 INFO CollectMultipleMetricsSpark - Initializing engine
21:00:58.791 INFO CollectMultipleMetricsSpark - Done initializing engine
21:00:58.791 INFO CollectMultipleMetricsSpark - Spark verbosity set to INFO (see --spark-verbosity argument)
21:00:58.792 INFO MemoryStore - Block broadcast_3361 stored as values in memory (estimated size 305.5 KiB, free 1912.5 MiB)
21:00:58.799 INFO MemoryStore - Block broadcast_3361_piece0 stored as bytes in memory (estimated size 64.1 KiB, free 1912.4 MiB)
21:00:58.799 INFO BlockManagerInfo - Added broadcast_3361_piece0 in memory on localhost:43253 (size: 64.1 KiB, free: 1918.1 MiB)
21:00:58.799 INFO SparkContext - Created broadcast 3361 from newAPIHadoopFile at PathSplitSource.java:96
21:00:58.819 INFO MemoryStore - Block broadcast_3362 stored as values in memory (estimated size 305.5 KiB, free 1912.1 MiB)
21:00:58.825 INFO MemoryStore - Block broadcast_3362_piece0 stored as bytes in memory (estimated size 64.1 KiB, free 1912.1 MiB)
21:00:58.825 INFO BlockManagerInfo - Added broadcast_3362_piece0 in memory on localhost:43253 (size: 64.1 KiB, free: 1918.1 MiB)
21:00:58.826 INFO SparkContext - Created broadcast 3362 from newAPIHadoopFile at PathSplitSource.java:96
21:00:58.845 INFO FileInputFormat - Total input files to process : 1
21:00:58.848 INFO SparkContext - Starting job: count at CollectMultipleMetricsSparkIntegrationTest.java:130
21:00:58.848 INFO DAGScheduler - Got job 877 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) with 1 output partitions
21:00:58.848 INFO DAGScheduler - Final stage: ResultStage 2463 (count at CollectMultipleMetricsSparkIntegrationTest.java:130)
21:00:58.848 INFO DAGScheduler - Parents of final stage: List()
21:00:58.848 INFO DAGScheduler - Missing parents: List()
21:00:58.849 INFO DAGScheduler - Submitting ResultStage 2463 (MapPartitionsRDD[10142] at filter at CollectMultipleMetricsSpark.java:193), which has no missing parents
21:00:58.876 INFO MemoryStore - Block broadcast_3363 stored as values in memory (estimated size 476.8 KiB, free 1911.6 MiB)
21:00:58.879 INFO MemoryStore - Block broadcast_3363_piece0 stored as bytes in memory (estimated size 207.4 KiB, free 1911.4 MiB)
21:00:58.879 INFO BlockManagerInfo - Added broadcast_3363_piece0 in memory on localhost:43253 (size: 207.4 KiB, free: 1917.9 MiB)
21:00:58.879 INFO SparkContext - Created broadcast 3363 from broadcast at DAGScheduler.scala:1580
21:00:58.879 INFO DAGScheduler - Submitting 1 missing tasks from ResultStage 2463 (MapPartitionsRDD[10142] at filter at CollectMultipleMetricsSpark.java:193) (first 15 tasks are for partitions Vector(0))
21:00:58.879 INFO TaskSchedulerImpl - Adding task set 2463.0 with 1 tasks resource profile 0
21:00:58.879 INFO TaskSetManager - Starting task 0.0 in stage 2463.0 (TID 1882) (localhost, executor driver, partition 0, PROCESS_LOCAL, 9688 bytes)
21:00:58.880 INFO Executor - Running task 0.0 in stage 2463.0 (TID 1882)
21:00:58.907 INFO NewHadoopRDD - Input split: file:/gatkCloneMountPoint/src/test/resources/org/broadinstitute/hellbender/metrics/analysis/CollectInsertSizeMetrics/insert_size_metrics_test.bam:0+8071
21:00:58.910 INFO Executor - Finished task 0.0 in stage 2463.0 (TID 1882). 989 bytes result sent to driver
21:00:58.911 INFO TaskSetManager - Finished task 0.0 in stage 2463.0 (TID 1882) in 31 ms on localhost (executor driver) (1/1)
21:00:58.911 INFO TaskSchedulerImpl - Removed TaskSet 2463.0, whose tasks have all completed, from pool
21:00:58.911 INFO DAGScheduler - ResultStage 2463 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) finished in 0.062 s
21:00:58.911 INFO DAGScheduler - Job 877 is finished. Cancelling potential speculative or zombie tasks for this job
21:00:58.911 INFO TaskSchedulerImpl - Killing all running tasks in stage 2463: Stage finished
21:00:58.911 INFO DAGScheduler - Job 877 finished: count at CollectMultipleMetricsSparkIntegrationTest.java:130, took 0.062585 s
21:00:58.911 INFO CollectMultipleMetricsSpark - Shutting down engine
[June 12, 2025 at 9:00:58 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1535115264
21:00:58.930 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
21:00:58.930 INFO CollectMultipleMetricsSpark - The Genome Analysis Toolkit (GATK) v4.6.2.0-8-g092bdf2-SNAPSHOT
21:00:58.930 INFO CollectMultipleMetricsSpark - For support and documentation go to https://software.broadinstitute.org/gatk/
21:00:58.930 INFO CollectMultipleMetricsSpark - Executing as root@e24cea241624 on Linux v6.11.0-1015-azure amd64
21:00:58.930 INFO CollectMultipleMetricsSpark - Java runtime: OpenJDK 64-Bit Server VM v17.0.12+7-Ubuntu-1ubuntu222.04
21:00:58.930 INFO CollectMultipleMetricsSpark - Start Date/Time: June 12, 2025 at 9:00:58 PM GMT
21:00:58.930 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
21:00:58.930 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
21:00:58.930 INFO CollectMultipleMetricsSpark - HTSJDK Version: 4.2.0
21:00:58.930 INFO CollectMultipleMetricsSpark - Picard Version: 3.4.0
21:00:58.930 INFO CollectMultipleMetricsSpark - Built for Spark Version: 3.5.0
21:00:58.930 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.COMPRESSION_LEVEL : 2
21:00:58.930 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
21:00:58.930 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
21:00:58.930 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
21:00:58.930 INFO CollectMultipleMetricsSpark - Deflater: IntelDeflater
21:00:58.930 INFO CollectMultipleMetricsSpark - Inflater: IntelInflater
21:00:58.930 INFO CollectMultipleMetricsSpark - GCS max retries/reopens: 20
21:00:58.930 INFO CollectMultipleMetricsSpark - Requester pays: disabled
21:00:58.930 WARN CollectMultipleMetricsSpark -
?[1m?[31m !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Warning: CollectMultipleMetricsSpark is a BETA tool and is not yet ready for use in production
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!?[0m
21:00:58.931 INFO CollectMultipleMetricsSpark - Initializing engine
21:00:58.931 INFO CollectMultipleMetricsSpark - Done initializing engine
21:00:58.931 INFO CollectMultipleMetricsSpark - Spark verbosity set to INFO (see --spark-verbosity argument)
21:00:58.933 INFO MemoryStore - Block broadcast_3364 stored as values in memory (estimated size 592.0 B, free 1911.4 MiB)
21:00:58.934 INFO MemoryStore - Block broadcast_3364_piece0 stored as bytes in memory (estimated size 209.0 B, free 1911.4 MiB)
21:00:58.934 INFO BlockManagerInfo - Added broadcast_3364_piece0 in memory on localhost:43253 (size: 209.0 B, free: 1917.9 MiB)
21:00:58.934 INFO SparkContext - Created broadcast 3364 from broadcast at CramSource.java:114
21:00:58.935 INFO MemoryStore - Block broadcast_3365 stored as values in memory (estimated size 305.5 KiB, free 1911.1 MiB)
21:00:58.946 INFO MemoryStore - Block broadcast_3365_piece0 stored as bytes in memory (estimated size 64.0 KiB, free 1911.0 MiB)
21:00:58.946 INFO BlockManagerInfo - Added broadcast_3365_piece0 in memory on localhost:43253 (size: 64.0 KiB, free: 1917.8 MiB)
21:00:58.947 INFO SparkContext - Created broadcast 3365 from newAPIHadoopFile at PathSplitSource.java:96
21:00:58.966 INFO MemoryStore - Block broadcast_3366 stored as values in memory (estimated size 592.0 B, free 1911.0 MiB)
21:00:58.966 INFO MemoryStore - Block broadcast_3366_piece0 stored as bytes in memory (estimated size 209.0 B, free 1911.0 MiB)
21:00:58.966 INFO BlockManagerInfo - Added broadcast_3366_piece0 in memory on localhost:43253 (size: 209.0 B, free: 1917.8 MiB)
21:00:58.967 INFO SparkContext - Created broadcast 3366 from broadcast at CramSource.java:114
21:00:58.967 INFO MemoryStore - Block broadcast_3367 stored as values in memory (estimated size 305.5 KiB, free 1910.7 MiB)
21:00:58.973 INFO MemoryStore - Block broadcast_3367_piece0 stored as bytes in memory (estimated size 64.0 KiB, free 1910.7 MiB)
21:00:58.973 INFO BlockManagerInfo - Added broadcast_3367_piece0 in memory on localhost:43253 (size: 64.0 KiB, free: 1917.7 MiB)
21:00:58.974 INFO SparkContext - Created broadcast 3367 from newAPIHadoopFile at PathSplitSource.java:96
21:00:58.982 INFO BlockManagerInfo - Removed broadcast_3348_piece0 on localhost:43253 in memory (size: 207.9 KiB, free: 1917.9 MiB)
21:00:58.982 INFO BlockManagerInfo - Removed broadcast_3340_piece0 on localhost:43253 in memory (size: 64.1 KiB, free: 1918.0 MiB)
21:00:58.983 INFO BlockManagerInfo - Removed broadcast_3341_piece0 on localhost:43253 in memory (size: 16.5 KiB, free: 1918.0 MiB)
21:00:58.983 INFO BlockManagerInfo - Removed broadcast_3346_piece0 on localhost:43253 in memory (size: 209.3 KiB, free: 1918.2 MiB)
21:00:58.983 INFO BlockManagerInfo - Removed broadcast_3357_piece0 on localhost:43253 in memory (size: 64.1 KiB, free: 1918.3 MiB)
21:00:58.983 INFO BlockManagerInfo - Removed broadcast_3342_piece0 on localhost:43253 in memory (size: 19.0 KiB, free: 1918.3 MiB)
21:00:58.984 INFO BlockManager - Removing RDD 10117
21:00:58.984 INFO BlockManagerInfo - Removed broadcast_3338_piece0 on localhost:43253 in memory (size: 64.1 KiB, free: 1918.4 MiB)
21:00:58.985 INFO BlockManagerInfo - Removed broadcast_3358_piece0 on localhost:43253 in memory (size: 8.4 KiB, free: 1918.4 MiB)
21:00:58.985 INFO BlockManager - Removing RDD 10104
21:00:58.985 INFO BlockManagerInfo - Removed broadcast_3355_piece0 on localhost:43253 in memory (size: 143.8 KiB, free: 1918.6 MiB)
21:00:58.986 INFO BlockManagerInfo - Removed broadcast_3353_piece0 on localhost:43253 in memory (size: 145.1 KiB, free: 1918.7 MiB)
21:00:58.986 INFO BlockManagerInfo - Removed broadcast_3363_piece0 on localhost:43253 in memory (size: 207.4 KiB, free: 1918.9 MiB)
21:00:58.986 INFO BlockManagerInfo - Removed broadcast_3349_piece0 on localhost:43253 in memory (size: 209.0 B, free: 1918.9 MiB)
21:00:58.987 INFO BlockManagerInfo - Removed broadcast_3343_piece0 on localhost:43253 in memory (size: 15.2 KiB, free: 1918.9 MiB)
21:00:58.988 INFO BlockManagerInfo - Removed broadcast_3361_piece0 on localhost:43253 in memory (size: 64.1 KiB, free: 1919.0 MiB)
21:00:58.988 INFO BlockManagerInfo - Removed broadcast_3354_piece0 on localhost:43253 in memory (size: 147.5 KiB, free: 1919.1 MiB)
21:00:58.988 INFO BlockManager - Removing RDD 10089
21:00:58.989 INFO BlockManagerInfo - Removed broadcast_3360_piece0 on localhost:43253 in memory (size: 14.6 KiB, free: 1919.2 MiB)
21:00:58.989 INFO BlockManagerInfo - Removed broadcast_3351_piece0 on localhost:43253 in memory (size: 209.0 B, free: 1919.2 MiB)
21:00:58.989 INFO BlockManagerInfo - Removed broadcast_3356_piece0 on localhost:43253 in memory (size: 8.4 KiB, free: 1919.2 MiB)
21:00:58.990 INFO BlockManagerInfo - Removed broadcast_3347_piece0 on localhost:43253 in memory (size: 211.9 KiB, free: 1919.4 MiB)
21:00:58.990 INFO BlockManagerInfo - Removed broadcast_3362_piece0 on localhost:43253 in memory (size: 64.1 KiB, free: 1919.5 MiB)
21:00:58.990 INFO BlockManagerInfo - Removed broadcast_3344_piece0 on localhost:43253 in memory (size: 64.1 KiB, free: 1919.5 MiB)
21:00:58.991 INFO BlockManagerInfo - Removed broadcast_3350_piece0 on localhost:43253 in memory (size: 64.0 KiB, free: 1919.6 MiB)
21:00:58.991 INFO BlockManagerInfo - Removed broadcast_3345_piece0 on localhost:43253 in memory (size: 64.1 KiB, free: 1919.7 MiB)
21:00:58.991 INFO BlockManagerInfo - Removed broadcast_3337_piece0 on localhost:43253 in memory (size: 8.4 KiB, free: 1919.7 MiB)
21:00:58.991 INFO BlockManagerInfo - Removed broadcast_3365_piece0 on localhost:43253 in memory (size: 64.0 KiB, free: 1919.7 MiB)
21:00:58.992 INFO BlockManagerInfo - Removed broadcast_3359_piece0 on localhost:43253 in memory (size: 64.1 KiB, free: 1919.8 MiB)
21:00:58.992 INFO BlockManagerInfo - Removed broadcast_3364_piece0 on localhost:43253 in memory (size: 209.0 B, free: 1919.8 MiB)
21:00:58.992 INFO BlockManagerInfo - Removed broadcast_3352_piece0 on localhost:43253 in memory (size: 64.0 KiB, free: 1919.9 MiB)
21:00:58.992 INFO BlockManagerInfo - Removed broadcast_3339_piece0 on localhost:43253 in memory (size: 8.4 KiB, free: 1919.9 MiB)
21:00:59.000 INFO FileInputFormat - Total input files to process : 1
21:00:59.004 INFO SparkContext - Starting job: count at CollectMultipleMetricsSparkIntegrationTest.java:130
21:00:59.004 INFO DAGScheduler - Got job 878 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) with 1 output partitions
21:00:59.004 INFO DAGScheduler - Final stage: ResultStage 2464 (count at CollectMultipleMetricsSparkIntegrationTest.java:130)
21:00:59.004 INFO DAGScheduler - Parents of final stage: List()
21:00:59.004 INFO DAGScheduler - Missing parents: List()
21:00:59.004 INFO DAGScheduler - Submitting ResultStage 2464 (MapPartitionsRDD[10153] at filter at CollectMultipleMetricsSpark.java:193), which has no missing parents
21:00:59.016 INFO MemoryStore - Block broadcast_3368 stored as values in memory (estimated size 329.7 KiB, free 1918.9 MiB)
21:00:59.018 INFO MemoryStore - Block broadcast_3368_piece0 stored as bytes in memory (estimated size 143.3 KiB, free 1918.8 MiB)
21:00:59.018 INFO BlockManagerInfo - Added broadcast_3368_piece0 in memory on localhost:43253 (size: 143.3 KiB, free: 1919.7 MiB)
21:00:59.018 INFO SparkContext - Created broadcast 3368 from broadcast at DAGScheduler.scala:1580
21:00:59.018 INFO DAGScheduler - Submitting 1 missing tasks from ResultStage 2464 (MapPartitionsRDD[10153] at filter at CollectMultipleMetricsSpark.java:193) (first 15 tasks are for partitions Vector(0))
21:00:59.019 INFO TaskSchedulerImpl - Adding task set 2464.0 with 1 tasks resource profile 0
21:00:59.019 INFO TaskSetManager - Starting task 0.0 in stage 2464.0 (TID 1883) (localhost, executor driver, partition 0, PROCESS_LOCAL, 9689 bytes)
21:00:59.019 INFO Executor - Running task 0.0 in stage 2464.0 (TID 1883)
21:00:59.044 INFO NewHadoopRDD - Input split: file:/gatkCloneMountPoint/src/test/resources/org/broadinstitute/hellbender/metrics/analysis/CollectInsertSizeMetrics/insert_size_metrics_test.cram:0+8617
21:00:59.048 INFO Executor - Finished task 0.0 in stage 2464.0 (TID 1883). 989 bytes result sent to driver
21:00:59.049 INFO TaskSetManager - Finished task 0.0 in stage 2464.0 (TID 1883) in 30 ms on localhost (executor driver) (1/1)
21:00:59.049 INFO TaskSchedulerImpl - Removed TaskSet 2464.0, whose tasks have all completed, from pool
21:00:59.049 INFO DAGScheduler - ResultStage 2464 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) finished in 0.045 s
21:00:59.049 INFO DAGScheduler - Job 878 is finished. Cancelling potential speculative or zombie tasks for this job
21:00:59.049 INFO TaskSchedulerImpl - Killing all running tasks in stage 2464: Stage finished
21:00:59.049 INFO DAGScheduler - Job 878 finished: count at CollectMultipleMetricsSparkIntegrationTest.java:130, took 0.045354 s
21:00:59.049 INFO CollectMultipleMetricsSpark - Shutting down engine
[June 12, 2025 at 9:00:59 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1535115264