[SPARK-53349][SQL] Optimized XML parser can't handle corrupted files correctly

[xiaonanyang-db]

What changes were proposed in this pull request?

In #51287, we introduced an optimized XML parser, which is more memory-efficient. However, the new parser reads the input stream eagerly on initialization. If the file is corrupted, the error is not caught properly and handled based on the ignoreCorruptedFiles option. This PR addresses the issue.

Why are the changes needed?

Bug fix

Does this PR introduce any user-facing change?

No

How was this patch tested?

New tests.

Was this patch authored or co-authored using generative AI tooling?

[sandip-db]

Instead of catching the same errors in multiple places, can you defer the reader initialization?

override def hasNext: Boolean = {
  if (reader == null) {
     reader = StaxXMLRecordReader(inputStream, options)
  }
   reader.hasMoreRecord
}

[xiaonanyang-db]

We can defer the initialization in this way. The hasNext function will be called outside before the actual row parsing, so the error won't be caught in this approach as well.

[xiaonanyang-db]

I have updated the code to consolidate the error handling of corrupt/missing files for parser initialization here

[xiaonanyang-db]

Actually, I found another way to defer the initialization. Updated the PR with the changes, PTAL

[HyukjinKwon]

Let's make the CI result green

[HyukjinKwon]

Merged to master.