Spark 3 129 .x followed Hive 129 's forgiving rules: a malformed cast returned NULL and integer overflow wrapped around, so '12x' could load as a NULL quantity unnoticed. Spark 4.0 sets spark.sql.ansi.enabled=true: on Spark 4.2, SELECT CAST('12x' AS INT) fails with [CAST_INVALID_INPUT] The value '12x' of the type "STRING" cannot be cast to "INT" because it is malformed (SQLSTATE 22018), and SELECT 2147483647 + 1 fails with [ARITHMETIC_OVERFLOW] and a hint to use try_add.
Where you expect dirty input, say so: try_cast('12x' AS INT) and try_add(2147483647, 1) return NULL, and the NULL count becomes a quality metric (Lakehouses, Data Quality and Governance). Treat each new error in an upgraded 3.x job as a data bug, not a reason to set spark.sql.ansi.enabled=false.