Skip to content

Java: convert all qlref tests to inline expectation tests using postprocessing#21966

Open
owen-mc wants to merge 5 commits into
github:mainfrom
owen-mc:java/convert-to-inline-expectation-tests
Open

Java: convert all qlref tests to inline expectation tests using postprocessing#21966
owen-mc wants to merge 5 commits into
github:mainfrom
owen-mc:java/convert-to-inline-expectation-tests

Conversation

@owen-mc

@owen-mc owen-mc commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@owen-mc owen-mc added the no-change-note-required This PR does not need a change note label Jun 10, 2026
@github-actions github-actions Bot added the Java label Jun 10, 2026
@owen-mc owen-mc marked this pull request as ready for review June 10, 2026 11:33
@owen-mc owen-mc requested review from a team as code owners June 10, 2026 11:33
Copilot AI review requested due to automatic review settings June 10, 2026 11:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

@mbaluda mbaluda left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what your scope is, but there are a lot missing in the other java folders:
grep -rL "InlineExpectationsTestQuery" --include="*.qlref" java

@owen-mc

owen-mc commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Don't speak to me, speak to copilot 😆 . I'll look into it. I think the scope is any .qlref test referencing any query that has @kind problem or @kind path-problem, and where it doesn't make the test unreadable.

@owen-mc

owen-mc commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

It was just looking in java/ql/test/ 🤦🏻 . I've run it on the rest now. It gave these reasons why it skipped some:

Java Inline Expectation Skip List

This file lists the remaining .qlref tests under java that intentionally do not use utils/test/InlineExpectationsTestQuery.ql.

AST dump queries

These queries produce AST or extractor structure dumps rather than ordinary @kind problem or @kind path-problem results.

  • java/ql/integration-tests/java/buildless-module-definition-not-in-module-info-file/PrintAst.qlref
  • java/ql/integration-tests/kotlin/all-platforms/annotation-id-consistency/PrintAst.qlref
  • java/ql/integration-tests/kotlin/all-platforms/gradle_kotlinx_serialization/PrintAst.qlref
  • java/ql/integration-tests/kotlin/linux/custom_plugin/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/annotation-accessor-result-type/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/annotation_classes/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/classes/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/collection-literals/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/data-classes/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/exprs/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/exprs_typeaccess/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/generics/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/java-map-methods/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/java_properties/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/jvmoverloads-annotation/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/jvmstatic-annotation/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/lateinit/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/operator-overloads/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/parameter-defaults/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/reflection/PrintAst.qlref
  • java/ql/test-kotlin1/library-tests/stmts/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/annotation-accessor-result-type/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/annotation_classes/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/classes/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/collection-literals/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/data-classes/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/exprs/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/exprs_typeaccess/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/generics/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/java-map-methods/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/java_properties/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/jvmoverloads-annotation/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/jvmstatic-annotation/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/lateinit/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/operator-overloads/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/parameter-defaults/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/reflection/PrintAst.qlref
  • java/ql/test-kotlin2/library-tests/stmts/PrintAst.qlref
  • java/ql/test/library-tests/JDK/PrintAst.qlref
  • java/ql/test/library-tests/arrays/PrintAst.qlref
  • java/ql/test/library-tests/collections/PrintAst.qlref
  • java/ql/test/library-tests/comment-encoding/PrintAst.qlref
  • java/ql/test/library-tests/comments/PrintAst.qlref
  • java/ql/test/library-tests/compact-source-files/PrintAst.qlref
  • java/ql/test/library-tests/constants/PrintAst.qlref
  • java/ql/test/library-tests/constructors/PrintAst.qlref
  • java/ql/test/library-tests/dependency-counts/PrintAst.qlref
  • java/ql/test/library-tests/dependency/PrintAst.qlref
  • java/ql/test/library-tests/errorexpr/PrintAst.qlref
  • java/ql/test/library-tests/errortype-with-params/PrintAst.qlref
  • java/ql/test/library-tests/errortype/PrintAst.qlref
  • java/ql/test/library-tests/fields/PrintAst.qlref
  • java/ql/test/library-tests/flexible-constructors/PrintAst.qlref
  • java/ql/test/library-tests/generics/PrintAst.qlref
  • java/ql/test/library-tests/guards12/PrintAst.qlref
  • java/ql/test/library-tests/java7/Diamond/PrintAst.qlref
  • java/ql/test/library-tests/java7/MultiCatch/PrintAst.qlref
  • java/ql/test/library-tests/javadoc/PrintAst.qlref
  • java/ql/test/library-tests/modifiers/PrintAst.qlref
  • java/ql/test/library-tests/pattern-instanceof/PrintAst.qlref
  • java/ql/test/library-tests/printAst/PrintAst.qlref
  • java/ql/test/library-tests/reflection/PrintAst.qlref
  • java/ql/test/library-tests/typeaccesses/PrintAst.qlref
  • java/ql/test/library-tests/varargs/PrintAst.qlref

Diagnostics and extractor telemetry

These queries validate extractor, database, or diagnostic output rather than source-level alerts.

  • java/ql/integration-tests/java/buildless-erroneous/DatabaseQualityDiagnostics.qlref
  • java/ql/integration-tests/java/buildless-erroneous/ExtractorInformation.qlref
  • java/ql/integration-tests/java/lambda-expression-buildless-recovery/ExtractionErrors.qlref
  • java/ql/integration-tests/java/maven_3_fetch_maven_4_wrapper/ExtractionErrors.qlref
  • java/ql/integration-tests/java/multi-release-jar-java11/ExtractorInformation.qlref
  • java/ql/integration-tests/java/multi-release-jar-java17/ExtractorInformation.qlref
  • java/ql/integration-tests/kotlin/all-platforms/extractor_information_kotlin1/ExtractorInformation.qlref
  • java/ql/integration-tests/kotlin/all-platforms/extractor_information_kotlin2/ExtractorInformation.qlref
  • java/ql/test/query-tests/Telemetry/ExternalLibraryUsage/ExternalLibraryUsage.qlref
  • java/ql/test/query-tests/Telemetry/SupportedExternalApis/SupportedExternalApis.qlref
  • java/ql/test/query-tests/Telemetry/SupportedExternalSinks/SupportedExternalSinks.qlref
  • java/ql/test/query-tests/Telemetry/SupportedExternalSources/SupportedExternalSources.qlref
  • java/ql/test/query-tests/Telemetry/SupportedExternalTaint/SupportedExternalTaint.qlref
  • java/ql/test/query-tests/Telemetry/UnsupportedExternalAPIs/UnsupportedExternalAPIs.qlref

Metrics, table, definitions, and tool-specific queries

These queries are not ordinary inline-alert candidates because they produce summary, table, definitions, stub-generation, or tool endpoint output.

  • java/ql/test/query-tests/Metrics/GeneratedVsManualCoverage/ApacheCommonsIoTest/GeneratedVsManualCoverage.qlref
  • java/ql/test/query-tests/Metrics/LinesOfCode/LinesOfCode.qlref
  • java/ql/test/query-tests/Stubs/Minimal/MinimalStubsFromSource.qlref
  • java/ql/test/query-tests/definitions/definitions.qlref
  • java/ql/test/query-tests/security/CWE-020/ExternalAPIsUsedWithUntrustedData.qlref
  • java/ql/test/utils/modeleditor/ApplicationModeEndpoints.qlref
  • java/ql/test/utils/modeleditor/FrameworkModeEndpoints.qlref

Special query kind

These tests do not use standard @kind problem or @kind path-problem inline expectations.

  • java/ql/test/query-tests/AlertSuppression/AlertSuppression.qlref
  • java/ql/test/query-tests/AlertSuppression/AlertSuppressionAnnotations.qlref

Alert location is inside Javadoc or literal content

Adding inline comments would change the text or syntax being reported, making the converted test misleading or brittle.

  • java/ql/test/query-tests/Javadoc/ImpossibleJavadocThrows.qlref
  • java/ql/test/query-tests/NonExplicitControlAndWhitespaceCharsInLiterals/NonExplicitControlAndWhitespaceCharsInLiterals.qlref
  • java/ql/test/query-tests/SpuriousJavadocParam/test.qlref

Results are in shared stubs or vendored fixtures

These dead-code results are primarily reported in shared Apache Camel stub files, so inline comments would pollute shared fixtures.

  • java/ql/test/query-tests/DeadCode/camel/DeadClass.qlref
  • java/ql/test/query-tests/DeadCode/camel/DeadMethod.qlref

@owen-mc owen-mc requested a review from a team as a code owner June 11, 2026 19:53
@owen-mc owen-mc force-pushed the java/convert-to-inline-expectation-tests branch from 37e782e to a4bf2b8 Compare June 11, 2026 21:00

@aschackmull aschackmull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Java Kotlin no-change-note-required This PR does not need a change note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants