# Changelog

## Baseline

- First survey — no prior run to compare against. CAI 43.3.

## Lenses

- Code Health 29.7
- Architecture 94.9
- Maturity 63.3
- Readiness 50.3
- Security 66.5
- Accessibility 46.7
- Performance 100.0

## Changes since last survey

- 300 commits — 297 feature/other, 3 fixes

## By area

- (root) — 270 commits
- core/src — 7 commits
- sql/core — 6 commits
- sql/catalyst — 5 commits
- mllib/src — 3 commits
- python/pyspark — 2 commits
- dev/.rat-excludes — 1 commit
- dev/run-tests.py — 1 commit
- dev/sparktestsupport — 1 commit
- resource-managers/kubernetes — 1 commit
- resource-managers/yarn — 1 commit
- sbin/start-slave.sh — 1 commit
- sql/hive-thriftserver — 1 commit

## Notable commits

- fix: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG
- fix: [SPARK-28154][ML][FOLLOWUP] GMM fix double caching
- fix: [SPARK-28164] Fix usage description of `start-slave.sh`
- change: [MINOR][BUILD] Exclude pyspark-coverage-site/ dir from RAT
- change: [SPARK-11412][SQL] Support merge schema for ORC
- change: [SPARK-23179][SQL] Support option to throw exception if overflow occurs during Decimal arithmetic
- change: [SPARK-27369][CORE] Setup resources when Standalone Worker starts up
- change: [SPARK-27622][CORE] Avoiding the network when block manager fetches disk persisted RDD blocks from the same host
- change: [SPARK-27630][CORE] Properly handle task end events from completed stages
- change: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
- change: [SPARK-27823][CORE][DOC][FOLLOWUP] Update doc of config `spark.driver.resourcesFile`
- change: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs
- change: [SPARK-27893][SQL][PYTHON][FOLLOW-UP] Allow Scalar Pandas and Python UDFs can be tested with Scala test base
- change: [SPARK-27945][SQL] Minimal changes to support columnar processing
- change: [SPARK-27992][PYTHON] Allow Python to join with connection thread to propagate errors
- change: [SPARK-28005][YARN] Remove unnecessary log from SparkRackResolver
- change: [SPARK-28045][ML][PYTHON] add missing RankingEvaluator
- change: [SPARK-28056][.2][PYTHON][SQL] add docstring/doctest for SCALAR_ITER Pandas UDF
- change: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
- change: [SPARK-28117][ML] LDA and BisectingKMeans cache the input dataset if necessary
- …and 280 more

## Architecture

- 0 containers · 10 bounded contexts · 7 dependency edges (baseline)
