[May 27, 2025 at 4:01:22 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1784676352
[May 27, 2025 at 4:01:22 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1784676352
[May 27, 2025 at 4:01:22 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1784676352
16:01:22.778 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
16:01:22.778 INFO CollectMultipleMetricsSpark - The Genome Analysis Toolkit (GATK) v4.6.2.0-6-gf5385f1-SNAPSHOT
16:01:22.778 INFO CollectMultipleMetricsSpark - For support and documentation go to https://software.broadinstitute.org/gatk/
16:01:22.778 INFO CollectMultipleMetricsSpark - Executing as root@1be272c3efa7 on Linux v6.11.0-1014-azure amd64
16:01:22.778 INFO CollectMultipleMetricsSpark - Java runtime: OpenJDK 64-Bit Server VM v17.0.12+7-Ubuntu-1ubuntu222.04
16:01:22.778 INFO CollectMultipleMetricsSpark - Start Date/Time: May 27, 2025 at 4:01:22 PM GMT
16:01:22.778 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
16:01:22.778 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
16:01:22.779 INFO CollectMultipleMetricsSpark - HTSJDK Version: test_cram_wiring-SNAPSHOT
16:01:22.779 INFO CollectMultipleMetricsSpark - Picard Version: 3.4.0
16:01:22.779 INFO CollectMultipleMetricsSpark - Built for Spark Version: 3.5.0
16:01:22.779 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.COMPRESSION_LEVEL : 2
16:01:22.779 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
16:01:22.779 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
16:01:22.779 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
16:01:22.779 INFO CollectMultipleMetricsSpark - Deflater: IntelDeflater
16:01:22.779 INFO CollectMultipleMetricsSpark - Inflater: IntelInflater
16:01:22.779 INFO CollectMultipleMetricsSpark - GCS max retries/reopens: 20
16:01:22.779 INFO CollectMultipleMetricsSpark - Requester pays: disabled
16:01:22.779 WARN CollectMultipleMetricsSpark -
?[1m?[31m !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Warning: CollectMultipleMetricsSpark is a BETA tool and is not yet ready for use in production
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!?[0m
16:01:22.779 INFO CollectMultipleMetricsSpark - Initializing engine
16:01:22.779 INFO CollectMultipleMetricsSpark - Done initializing engine
16:01:22.779 INFO CollectMultipleMetricsSpark - Spark verbosity set to INFO (see --spark-verbosity argument)
WARNING 2025-05-27 16:01:22 SamReaderFactory Unable to detect file format from input URL or stream, assuming SAM format.
WARNING 2025-05-27 16:01:22 SamReaderFactory Unable to detect file format from input URL or stream, assuming SAM format.
16:01:22.780 INFO MemoryStore - Block broadcast_3374 stored as values in memory (estimated size 37.8 KiB, free 1908.4 MiB)
16:01:22.781 INFO MemoryStore - Block broadcast_3374_piece0 stored as bytes in memory (estimated size 8.4 KiB, free 1908.4 MiB)
16:01:22.781 INFO BlockManagerInfo - Added broadcast_3374_piece0 in memory on localhost:41411 (size: 8.4 KiB, free: 1918.0 MiB)
16:01:22.781 INFO SparkContext - Created broadcast 3374 from broadcast at SamSource.java:78
16:01:22.782 INFO MemoryStore - Block broadcast_3375 stored as values in memory (estimated size 305.5 KiB, free 1908.1 MiB)
16:01:22.788 INFO MemoryStore - Block broadcast_3375_piece0 stored as bytes in memory (estimated size 64.1 KiB, free 1908.0 MiB)
16:01:22.788 INFO BlockManagerInfo - Added broadcast_3375_piece0 in memory on localhost:41411 (size: 64.1 KiB, free: 1917.9 MiB)
16:01:22.788 INFO SparkContext - Created broadcast 3375 from newAPIHadoopFile at SamSource.java:108
WARNING 2025-05-27 16:01:22 SamReaderFactory Unable to detect file format from input URL or stream, assuming SAM format.
WARNING 2025-05-27 16:01:22 SamReaderFactory Unable to detect file format from input URL or stream, assuming SAM format.
16:01:22.791 INFO MemoryStore - Block broadcast_3376 stored as values in memory (estimated size 37.8 KiB, free 1908.0 MiB)
16:01:22.791 INFO MemoryStore - Block broadcast_3376_piece0 stored as bytes in memory (estimated size 8.4 KiB, free 1907.9 MiB)
16:01:22.791 INFO BlockManagerInfo - Added broadcast_3376_piece0 in memory on localhost:41411 (size: 8.4 KiB, free: 1917.9 MiB)
16:01:22.791 INFO SparkContext - Created broadcast 3376 from broadcast at SamSource.java:78
16:01:22.792 INFO MemoryStore - Block broadcast_3377 stored as values in memory (estimated size 305.5 KiB, free 1907.6 MiB)
16:01:22.798 INFO MemoryStore - Block broadcast_3377_piece0 stored as bytes in memory (estimated size 64.1 KiB, free 1907.6 MiB)
16:01:22.798 INFO BlockManagerInfo - Added broadcast_3377_piece0 in memory on localhost:41411 (size: 64.1 KiB, free: 1917.8 MiB)
16:01:22.799 INFO SparkContext - Created broadcast 3377 from newAPIHadoopFile at SamSource.java:108
16:01:22.801 INFO FileInputFormat - Total input files to process : 1
16:01:22.804 INFO SparkContext - Starting job: count at CollectMultipleMetricsSparkIntegrationTest.java:130
16:01:22.804 INFO DAGScheduler - Got job 882 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) with 1 output partitions
16:01:22.804 INFO DAGScheduler - Final stage: ResultStage 2471 (count at CollectMultipleMetricsSparkIntegrationTest.java:130)
16:01:22.804 INFO DAGScheduler - Parents of final stage: List()
16:01:22.804 INFO DAGScheduler - Missing parents: List()
16:01:22.804 INFO DAGScheduler - Submitting ResultStage 2471 (MapPartitionsRDD[10156] at filter at CollectMultipleMetricsSpark.java:193), which has no missing parents
16:01:22.805 INFO MemoryStore - Block broadcast_3378 stored as values in memory (estimated size 34.8 KiB, free 1907.6 MiB)
16:01:22.805 INFO MemoryStore - Block broadcast_3378_piece0 stored as bytes in memory (estimated size 14.6 KiB, free 1907.5 MiB)
16:01:22.805 INFO BlockManagerInfo - Added broadcast_3378_piece0 in memory on localhost:41411 (size: 14.6 KiB, free: 1917.8 MiB)
16:01:22.805 INFO SparkContext - Created broadcast 3378 from broadcast at DAGScheduler.scala:1580
16:01:22.805 INFO DAGScheduler - Submitting 1 missing tasks from ResultStage 2471 (MapPartitionsRDD[10156] at filter at CollectMultipleMetricsSpark.java:193) (first 15 tasks are for partitions Vector(0))
16:01:22.805 INFO TaskSchedulerImpl - Adding task set 2471.0 with 1 tasks resource profile 0
16:01:22.806 INFO TaskSetManager - Starting task 0.0 in stage 2471.0 (TID 1890) (localhost, executor driver, partition 0, PROCESS_LOCAL, 9688 bytes)
16:01:22.806 INFO Executor - Running task 0.0 in stage 2471.0 (TID 1890)
16:01:22.807 INFO NewHadoopRDD - Input split: file:/gatkCloneMountPoint/src/test/resources/org/broadinstitute/hellbender/metrics/analysis/CollectInsertSizeMetrics/insert_size_metrics_test.sam:0+44008
16:01:22.809 INFO Executor - Finished task 0.0 in stage 2471.0 (TID 1890). 989 bytes result sent to driver
16:01:22.809 INFO TaskSetManager - Finished task 0.0 in stage 2471.0 (TID 1890) in 3 ms on localhost (executor driver) (1/1)
16:01:22.809 INFO TaskSchedulerImpl - Removed TaskSet 2471.0, whose tasks have all completed, from pool
16:01:22.809 INFO DAGScheduler - ResultStage 2471 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) finished in 0.005 s
16:01:22.809 INFO DAGScheduler - Job 882 is finished. Cancelling potential speculative or zombie tasks for this job
16:01:22.809 INFO TaskSchedulerImpl - Killing all running tasks in stage 2471: Stage finished
16:01:22.809 INFO DAGScheduler - Job 882 finished: count at CollectMultipleMetricsSparkIntegrationTest.java:130, took 0.005474 s
16:01:22.809 INFO CollectMultipleMetricsSpark - Shutting down engine
[May 27, 2025 at 4:01:22 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1784676352
16:01:22.823 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
16:01:22.823 INFO CollectMultipleMetricsSpark - The Genome Analysis Toolkit (GATK) v4.6.2.0-6-gf5385f1-SNAPSHOT
16:01:22.823 INFO CollectMultipleMetricsSpark - For support and documentation go to https://software.broadinstitute.org/gatk/
16:01:22.823 INFO CollectMultipleMetricsSpark - Executing as root@1be272c3efa7 on Linux v6.11.0-1014-azure amd64
16:01:22.823 INFO CollectMultipleMetricsSpark - Java runtime: OpenJDK 64-Bit Server VM v17.0.12+7-Ubuntu-1ubuntu222.04
16:01:22.823 INFO CollectMultipleMetricsSpark - Start Date/Time: May 27, 2025 at 4:01:22 PM GMT
16:01:22.823 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
16:01:22.823 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
16:01:22.824 INFO CollectMultipleMetricsSpark - HTSJDK Version: test_cram_wiring-SNAPSHOT
16:01:22.824 INFO CollectMultipleMetricsSpark - Picard Version: 3.4.0
16:01:22.824 INFO CollectMultipleMetricsSpark - Built for Spark Version: 3.5.0
16:01:22.824 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.COMPRESSION_LEVEL : 2
16:01:22.824 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
16:01:22.824 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
16:01:22.824 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
16:01:22.824 INFO CollectMultipleMetricsSpark - Deflater: IntelDeflater
16:01:22.824 INFO CollectMultipleMetricsSpark - Inflater: IntelInflater
16:01:22.824 INFO CollectMultipleMetricsSpark - GCS max retries/reopens: 20
16:01:22.824 INFO CollectMultipleMetricsSpark - Requester pays: disabled
16:01:22.824 WARN CollectMultipleMetricsSpark -
?[1m?[31m !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Warning: CollectMultipleMetricsSpark is a BETA tool and is not yet ready for use in production
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!?[0m
16:01:22.824 INFO CollectMultipleMetricsSpark - Initializing engine
16:01:22.824 INFO CollectMultipleMetricsSpark - Done initializing engine
16:01:22.824 INFO CollectMultipleMetricsSpark - Spark verbosity set to INFO (see --spark-verbosity argument)
16:01:22.825 INFO MemoryStore - Block broadcast_3379 stored as values in memory (estimated size 305.5 KiB, free 1907.2 MiB)
16:01:22.832 INFO MemoryStore - Block broadcast_3379_piece0 stored as bytes in memory (estimated size 64.1 KiB, free 1907.2 MiB)
16:01:22.832 INFO BlockManagerInfo - Added broadcast_3379_piece0 in memory on localhost:41411 (size: 64.1 KiB, free: 1917.7 MiB)
16:01:22.832 INFO SparkContext - Created broadcast 3379 from newAPIHadoopFile at PathSplitSource.java:96
16:01:22.852 INFO MemoryStore - Block broadcast_3380 stored as values in memory (estimated size 305.5 KiB, free 1906.9 MiB)
16:01:22.858 INFO MemoryStore - Block broadcast_3380_piece0 stored as bytes in memory (estimated size 64.1 KiB, free 1906.8 MiB)
16:01:22.858 INFO BlockManagerInfo - Added broadcast_3380_piece0 in memory on localhost:41411 (size: 64.1 KiB, free: 1917.7 MiB)
16:01:22.858 INFO SparkContext - Created broadcast 3380 from newAPIHadoopFile at PathSplitSource.java:96
16:01:22.878 INFO FileInputFormat - Total input files to process : 1
16:01:22.881 INFO SparkContext - Starting job: count at CollectMultipleMetricsSparkIntegrationTest.java:130
16:01:22.881 INFO DAGScheduler - Got job 883 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) with 1 output partitions
16:01:22.881 INFO DAGScheduler - Final stage: ResultStage 2472 (count at CollectMultipleMetricsSparkIntegrationTest.java:130)
16:01:22.881 INFO DAGScheduler - Parents of final stage: List()
16:01:22.881 INFO DAGScheduler - Missing parents: List()
16:01:22.881 INFO DAGScheduler - Submitting ResultStage 2472 (MapPartitionsRDD[10169] at filter at CollectMultipleMetricsSpark.java:193), which has no missing parents
16:01:22.898 INFO MemoryStore - Block broadcast_3381 stored as values in memory (estimated size 476.8 KiB, free 1906.3 MiB)
16:01:22.900 INFO MemoryStore - Block broadcast_3381_piece0 stored as bytes in memory (estimated size 207.4 KiB, free 1906.1 MiB)
16:01:22.900 INFO BlockManagerInfo - Added broadcast_3381_piece0 in memory on localhost:41411 (size: 207.4 KiB, free: 1917.5 MiB)
16:01:22.900 INFO SparkContext - Created broadcast 3381 from broadcast at DAGScheduler.scala:1580
16:01:22.900 INFO DAGScheduler - Submitting 1 missing tasks from ResultStage 2472 (MapPartitionsRDD[10169] at filter at CollectMultipleMetricsSpark.java:193) (first 15 tasks are for partitions Vector(0))
16:01:22.900 INFO TaskSchedulerImpl - Adding task set 2472.0 with 1 tasks resource profile 0
16:01:22.901 INFO TaskSetManager - Starting task 0.0 in stage 2472.0 (TID 1891) (localhost, executor driver, partition 0, PROCESS_LOCAL, 9688 bytes)
16:01:22.901 INFO Executor - Running task 0.0 in stage 2472.0 (TID 1891)
16:01:22.928 INFO NewHadoopRDD - Input split: file:/gatkCloneMountPoint/src/test/resources/org/broadinstitute/hellbender/metrics/analysis/CollectInsertSizeMetrics/insert_size_metrics_test.bam:0+8071
16:01:22.932 INFO Executor - Finished task 0.0 in stage 2472.0 (TID 1891). 989 bytes result sent to driver
16:01:22.932 INFO TaskSetManager - Finished task 0.0 in stage 2472.0 (TID 1891) in 31 ms on localhost (executor driver) (1/1)
16:01:22.932 INFO TaskSchedulerImpl - Removed TaskSet 2472.0, whose tasks have all completed, from pool
16:01:22.932 INFO DAGScheduler - ResultStage 2472 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) finished in 0.051 s
16:01:22.932 INFO DAGScheduler - Job 883 is finished. Cancelling potential speculative or zombie tasks for this job
16:01:22.932 INFO TaskSchedulerImpl - Killing all running tasks in stage 2472: Stage finished
16:01:22.932 INFO DAGScheduler - Job 883 finished: count at CollectMultipleMetricsSparkIntegrationTest.java:130, took 0.051635 s
16:01:22.932 INFO CollectMultipleMetricsSpark - Shutting down engine
[May 27, 2025 at 4:01:22 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1784676352
16:01:22.948 INFO BlockManagerInfo - Removed broadcast_3371_piece0 on localhost:41411 in memory (size: 145.1 KiB, free: 1917.6 MiB)
16:01:22.949 INFO BlockManagerInfo - Removed broadcast_3381_piece0 on localhost:41411 in memory (size: 207.4 KiB, free: 1917.8 MiB)
16:01:22.949 INFO BlockManagerInfo - Removed broadcast_3369_piece0 on localhost:41411 in memory (size: 209.0 B, free: 1917.8 MiB)
16:01:22.949 INFO BlockManagerInfo - Removed broadcast_3375_piece0 on localhost:41411 in memory (size: 64.1 KiB, free: 1917.9 MiB)
16:01:22.950 INFO BlockManagerInfo - Removed broadcast_3378_piece0 on localhost:41411 in memory (size: 14.6 KiB, free: 1917.9 MiB)
16:01:22.950 INFO BlockManagerInfo - Removed broadcast_3355_piece0 on localhost:41411 in memory (size: 8.4 KiB, free: 1917.9 MiB)
16:01:22.950 INFO BlockManagerInfo - Removed broadcast_3374_piece0 on localhost:41411 in memory (size: 8.4 KiB, free: 1917.9 MiB)
16:01:22.950 INFO BlockManagerInfo - Removed broadcast_3362_piece0 on localhost:41411 in memory (size: 64.1 KiB, free: 1918.0 MiB)
16:01:22.951 INFO BlockManagerInfo - Removed broadcast_3379_piece0 on localhost:41411 in memory (size: 64.1 KiB, free: 1918.0 MiB)
16:01:22.951 INFO BlockManagerInfo - Removed broadcast_3358_piece0 on localhost:41411 in memory (size: 64.1 KiB, free: 1918.1 MiB)
16:01:22.951 INFO BlockManagerInfo - Removed broadcast_3368_piece0 on localhost:41411 in memory (size: 64.0 KiB, free: 1918.2 MiB)
16:01:22.952 INFO BlockManagerInfo - Removed broadcast_3359_piece0 on localhost:41411 in memory (size: 16.5 KiB, free: 1918.2 MiB)
16:01:22.952 INFO BlockManagerInfo - Removed broadcast_3357_piece0 on localhost:41411 in memory (size: 8.4 KiB, free: 1918.2 MiB)
16:01:22.952 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
16:01:22.952 INFO CollectMultipleMetricsSpark - The Genome Analysis Toolkit (GATK) v4.6.2.0-6-gf5385f1-SNAPSHOT
16:01:22.952 INFO CollectMultipleMetricsSpark - For support and documentation go to https://software.broadinstitute.org/gatk/
16:01:22.952 INFO CollectMultipleMetricsSpark - Executing as root@1be272c3efa7 on Linux v6.11.0-1014-azure amd64
16:01:22.952 INFO CollectMultipleMetricsSpark - Java runtime: OpenJDK 64-Bit Server VM v17.0.12+7-Ubuntu-1ubuntu222.04
16:01:22.952 INFO CollectMultipleMetricsSpark - Start Date/Time: May 27, 2025 at 4:01:22 PM GMT
16:01:22.952 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
16:01:22.952 INFO CollectMultipleMetricsSpark - ------------------------------------------------------------
16:01:22.952 INFO CollectMultipleMetricsSpark - HTSJDK Version: test_cram_wiring-SNAPSHOT
16:01:22.952 INFO CollectMultipleMetricsSpark - Picard Version: 3.4.0
16:01:22.952 INFO CollectMultipleMetricsSpark - Built for Spark Version: 3.5.0
16:01:22.952 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.COMPRESSION_LEVEL : 2
16:01:22.952 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
16:01:22.952 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
16:01:22.952 INFO CollectMultipleMetricsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
16:01:22.952 INFO CollectMultipleMetricsSpark - Deflater: IntelDeflater
16:01:22.952 INFO CollectMultipleMetricsSpark - Inflater: IntelInflater
16:01:22.952 INFO CollectMultipleMetricsSpark - GCS max retries/reopens: 20
16:01:22.952 INFO CollectMultipleMetricsSpark - Requester pays: disabled
16:01:22.952 WARN CollectMultipleMetricsSpark -
?[1m?[31m !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Warning: CollectMultipleMetricsSpark is a BETA tool and is not yet ready for use in production
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!?[0m
16:01:22.952 INFO CollectMultipleMetricsSpark - Initializing engine
16:01:22.952 INFO CollectMultipleMetricsSpark - Done initializing engine
16:01:22.952 INFO CollectMultipleMetricsSpark - Spark verbosity set to INFO (see --spark-verbosity argument)
16:01:22.953 INFO BlockManagerInfo - Removed broadcast_3380_piece0 on localhost:41411 in memory (size: 64.1 KiB, free: 1918.3 MiB)
16:01:22.953 INFO BlockManagerInfo - Removed broadcast_3373_piece0 on localhost:41411 in memory (size: 143.8 KiB, free: 1918.4 MiB)
16:01:22.953 INFO BlockManager - Removing RDD 10131
16:01:22.954 INFO BlockManagerInfo - Removed broadcast_3367_piece0 on localhost:41411 in memory (size: 209.0 B, free: 1918.4 MiB)
16:01:22.954 INFO BlockManagerInfo - Removed broadcast_3364_piece0 on localhost:41411 in memory (size: 209.3 KiB, free: 1918.6 MiB)
16:01:22.954 INFO BlockManagerInfo - Removed broadcast_3356_piece0 on localhost:41411 in memory (size: 64.1 KiB, free: 1918.7 MiB)
16:01:22.955 INFO BlockManagerInfo - Removed broadcast_3360_piece0 on localhost:41411 in memory (size: 19.0 KiB, free: 1918.7 MiB)
16:01:22.955 INFO MemoryStore - Block broadcast_3382 stored as values in memory (estimated size 592.0 B, free 1911.3 MiB)
16:01:22.955 INFO BlockManager - Removing RDD 10144
16:01:22.955 INFO MemoryStore - Block broadcast_3382_piece0 stored as bytes in memory (estimated size 209.0 B, free 1911.3 MiB)
16:01:22.955 INFO BlockManagerInfo - Added broadcast_3382_piece0 in memory on localhost:41411 (size: 209.0 B, free: 1918.7 MiB)
16:01:22.955 INFO BlockManagerInfo - Removed broadcast_3372_piece0 on localhost:41411 in memory (size: 147.5 KiB, free: 1918.9 MiB)
16:01:22.956 INFO SparkContext - Created broadcast 3382 from broadcast at CramSource.java:114
16:01:22.956 INFO BlockManager - Removing RDD 10116
16:01:22.956 INFO BlockManagerInfo - Removed broadcast_3366_piece0 on localhost:41411 in memory (size: 207.9 KiB, free: 1919.1 MiB)
16:01:22.956 INFO BlockManagerInfo - Removed broadcast_3363_piece0 on localhost:41411 in memory (size: 64.1 KiB, free: 1919.2 MiB)
16:01:22.956 INFO MemoryStore - Block broadcast_3383 stored as values in memory (estimated size 305.5 KiB, free 1912.6 MiB)
16:01:22.957 INFO BlockManagerInfo - Removed broadcast_3370_piece0 on localhost:41411 in memory (size: 64.0 KiB, free: 1919.3 MiB)
16:01:22.957 INFO BlockManagerInfo - Removed broadcast_3377_piece0 on localhost:41411 in memory (size: 64.1 KiB, free: 1919.3 MiB)
16:01:22.957 INFO BlockManagerInfo - Removed broadcast_3361_piece0 on localhost:41411 in memory (size: 15.2 KiB, free: 1919.3 MiB)
16:01:22.958 INFO BlockManagerInfo - Removed broadcast_3376_piece0 on localhost:41411 in memory (size: 8.4 KiB, free: 1919.3 MiB)
16:01:22.958 INFO BlockManagerInfo - Removed broadcast_3365_piece0 on localhost:41411 in memory (size: 211.9 KiB, free: 1919.6 MiB)
16:01:22.964 INFO MemoryStore - Block broadcast_3383_piece0 stored as bytes in memory (estimated size 64.0 KiB, free 1914.0 MiB)
16:01:22.964 INFO BlockManagerInfo - Added broadcast_3383_piece0 in memory on localhost:41411 (size: 64.0 KiB, free: 1919.5 MiB)
16:01:22.964 INFO SparkContext - Created broadcast 3383 from newAPIHadoopFile at PathSplitSource.java:96
16:01:22.978 INFO MemoryStore - Block broadcast_3384 stored as values in memory (estimated size 592.0 B, free 1914.0 MiB)
16:01:22.978 INFO MemoryStore - Block broadcast_3384_piece0 stored as bytes in memory (estimated size 209.0 B, free 1914.0 MiB)
16:01:22.978 INFO BlockManagerInfo - Added broadcast_3384_piece0 in memory on localhost:41411 (size: 209.0 B, free: 1919.5 MiB)
16:01:22.979 INFO SparkContext - Created broadcast 3384 from broadcast at CramSource.java:114
16:01:22.979 INFO MemoryStore - Block broadcast_3385 stored as values in memory (estimated size 305.5 KiB, free 1913.7 MiB)
16:01:22.986 INFO MemoryStore - Block broadcast_3385_piece0 stored as bytes in memory (estimated size 64.0 KiB, free 1913.6 MiB)
16:01:22.986 INFO BlockManagerInfo - Added broadcast_3385_piece0 in memory on localhost:41411 (size: 64.0 KiB, free: 1919.4 MiB)
16:01:22.986 INFO SparkContext - Created broadcast 3385 from newAPIHadoopFile at PathSplitSource.java:96
16:01:23.000 INFO FileInputFormat - Total input files to process : 1
16:01:23.003 INFO SparkContext - Starting job: count at CollectMultipleMetricsSparkIntegrationTest.java:130
16:01:23.003 INFO DAGScheduler - Got job 884 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) with 1 output partitions
16:01:23.003 INFO DAGScheduler - Final stage: ResultStage 2473 (count at CollectMultipleMetricsSparkIntegrationTest.java:130)
16:01:23.003 INFO DAGScheduler - Parents of final stage: List()
16:01:23.003 INFO DAGScheduler - Missing parents: List()
16:01:23.003 INFO DAGScheduler - Submitting ResultStage 2473 (MapPartitionsRDD[10180] at filter at CollectMultipleMetricsSpark.java:193), which has no missing parents
16:01:23.015 INFO MemoryStore - Block broadcast_3386 stored as values in memory (estimated size 329.7 KiB, free 1913.3 MiB)
16:01:23.016 INFO MemoryStore - Block broadcast_3386_piece0 stored as bytes in memory (estimated size 143.3 KiB, free 1913.2 MiB)
16:01:23.016 INFO BlockManagerInfo - Added broadcast_3386_piece0 in memory on localhost:41411 (size: 143.3 KiB, free: 1919.3 MiB)
16:01:23.016 INFO SparkContext - Created broadcast 3386 from broadcast at DAGScheduler.scala:1580
16:01:23.017 INFO DAGScheduler - Submitting 1 missing tasks from ResultStage 2473 (MapPartitionsRDD[10180] at filter at CollectMultipleMetricsSpark.java:193) (first 15 tasks are for partitions Vector(0))
16:01:23.017 INFO TaskSchedulerImpl - Adding task set 2473.0 with 1 tasks resource profile 0
16:01:23.017 INFO TaskSetManager - Starting task 0.0 in stage 2473.0 (TID 1892) (localhost, executor driver, partition 0, PROCESS_LOCAL, 9689 bytes)
16:01:23.017 INFO Executor - Running task 0.0 in stage 2473.0 (TID 1892)
16:01:23.037 INFO NewHadoopRDD - Input split: file:/gatkCloneMountPoint/src/test/resources/org/broadinstitute/hellbender/metrics/analysis/CollectInsertSizeMetrics/insert_size_metrics_test.cram:0+8617
16:01:23.040 INFO Executor - Finished task 0.0 in stage 2473.0 (TID 1892). 989 bytes result sent to driver
16:01:23.041 INFO TaskSetManager - Finished task 0.0 in stage 2473.0 (TID 1892) in 24 ms on localhost (executor driver) (1/1)
16:01:23.041 INFO TaskSchedulerImpl - Removed TaskSet 2473.0, whose tasks have all completed, from pool
16:01:23.041 INFO DAGScheduler - ResultStage 2473 (count at CollectMultipleMetricsSparkIntegrationTest.java:130) finished in 0.038 s
16:01:23.041 INFO DAGScheduler - Job 884 is finished. Cancelling potential speculative or zombie tasks for this job
16:01:23.041 INFO TaskSchedulerImpl - Killing all running tasks in stage 2473: Stage finished
16:01:23.041 INFO DAGScheduler - Job 884 finished: count at CollectMultipleMetricsSparkIntegrationTest.java:130, took 0.038195 s
16:01:23.041 INFO CollectMultipleMetricsSpark - Shutting down engine
[May 27, 2025 at 4:01:23 PM GMT] org.broadinstitute.hellbender.tools.spark.pipelines.metrics.CollectMultipleMetricsSpark done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=1784676352