21-dec_2 - Google Drive REPACK
Dataproc Serverless for Spark runtime versions 1.0.22 and 2.0.2 will be deprecated on 11/11/2022. New batch submissions that use these runtime versions will fail starting 11/11/2022. This is due to an update to the google auth library which breaks running Pyspark batch workloads having dependency on gcsfs. Upcoming runtime versions will address this issue.
21-dec_2 - Google Drive
Dataproc images 2.0.50 and preview 2.1.0-RC3 are deprecated and cluster creations based on these images will fail starting 11/11/2022. This is due to an update to the google auth library which breaks running Pyspark batch workloads having dependency on gcsfs. Upcoming image versions will have a fix to address this issue.
Increase Dataproc agent buffer size for streaming driver output to Cloud Storage from 60KB to 8MB. This significantly increases streaming throughput: from 200KBps to 7MBps. However, it also increases the memory footprint for each job in the Dataproc agent from 100KB to 10MB.
Added additional information into Autoscaler recommendations in Stackdriver logging. In particular each recommendation now includes 1) the minimum and maximum worker counts configured in the autoscaling policy, 2) the graceful decommission timeout (for SCALE_DOWN operations), and 3) additional context into why the autoscaler couldn't add or remove more nodes. For example, it will indicate if it could not scale up further due to available regional or global quotas in the project (SCALING_CAPPED_DUE_TO_LACK_OF_QUOTA).
* Note: Schedules are subject to change due to poor weather, contractor schedules and other factors. Motorists are advised to observe signs in construction zones, follow directions from Flaggers and drive carefully.
Electronic convenience vehicles are available for rent inside the Visitor Center and at the Information Stations at Panda Plaza (parking lots A and B) and Lion-Tiger Hill (parking lot D). Please note: A state-issued driver's license is required to rent and operate an electronic convenience vehicle. 041b061a72