Update Java documentation links to Java 11; use `.inc` infix for included qhelp files #5226

Marcono1234 · 2021-02-19T20:08:51Z

Where possible update Java documentation links to Java 11.
Additionally update some other links to use HTTPS (see also #5163).

Notable additional changes:

Only upgraded guides to Java 8 because apparently a lot of the guides were not migrated to newer versions
Only upgraded Java-EE links to Java 7 because newer EE versions are not maintained by OpenJDK anymore
Updated Eclipse IDE links to 2020-12; use the link variant which does not show a navigation sidebar
Consistently used "Java API Specification" when referring to https://docs.oracle.com/en/java/javase/11/docs/api/index.html
- Removed Java version numbers because that would have to be manually changed again when links are upgraded
Replaced https://www.securecoding.cert.org with the https://wiki.sei.cmu.edu redirect

Marcono1234 · 2021-02-19T20:09:57Z

java/ql/src/Likely Bugs/Concurrency/DateFormatThreadUnsafe.qhelp

-<!-- LINKS NO LONGER VALID:
-<p>Sun Bug Report: <a href="http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6178997">Doc: it's not explicitly documented java.util.Calendar serialization are thread-unsafe</a></p>
-<p>Sun Bug Report: <a href="http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6231579">ArrayIndexOutOfBoundsException in BaseCalendar</a></p>
-->


Removed the links (even though the bug repots can still be found) because the bug reports are about bugs in the JDK source code, which is likely not relevant to most of the users.

Marcono1234 · 2021-02-19T20:11:29Z

java/ql/src/Likely Bugs/I18N/MissingLocaleArgument.qhelp

 for different locales. For example, if the default locale is ENGLISH, the function <code>toLowerCase()</code>
 converts a capital <code>I</code> to <code>i</code>; if the default locale is TURKISH, the function 
 <code>toLowerCase()</code> converts a capital <code>I</code> to the Unicode Character "Latin small 
-letter dotless i" (U+0131) (<a href="http://character-code.com/turkish-html-codes.php">Turkish HTML 


Removed this link because it does not work anymore, is also probably not needed.

Marcono1234 · 2021-02-19T20:12:50Z

java/ql/src/Violations of Best Practice/Undesirable Calls/DefaultToString.qhelp

-<li>
-  Java Language Specification, 3rd ed: <a href="https://docs.oracle.com/javase/specs/jls/se6/html/conversions.html#5.4">5.4 String Conversion</a>.


Removed this link because it is about String concatenation in genereal and not related to toString().

Marcono1234 · 2021-02-19T20:14:57Z

java/ql/src/Likely Bugs/Statements/ImpossibleCast.qhelp

-  <a href="http://docs.oracle.com/javase/specs/jls/se7/html/jls-5.html#jls-5.5.3">Checked Casts at Run Time</a>,
-  <a href="http://docs.oracle.com/javase/specs/jls/se7/html/jls-5.html#jls-5.5.1">Reference Type Casting</a>,
-  <a href="http://docs.oracle.com/javase/specs/jls/se7/html/jls-4.html#jls-4.10.3">Subtyping among Array Types</a>.
+  Java Language Specification:
+  <a href="https://docs.oracle.com/javase/specs/jls/se11/html/jls-5.html#jls-5.1.6">Narrowing Reference Conversion</a>,
+  <a href="https://docs.oracle.com/javase/specs/jls/se11/html/jls-4.html#jls-4.10.3">Subtyping among Array Types</a>.


These §5.5 sections do not exist anymore in the latest versions, it looks like the content of that section has changed a lot.

Marcono1234 · 2021-02-19T20:15:49Z

java/ql/src/Likely Bugs/Comparison/InconsistentCompareTo.qhelp

 <li>
-  Java API Documentation: 
-  <a href="http://docs.oracle.com/javase/6/docs/api/java/lang/Comparable.html#compareTo%28T%29">Comparable.compareTo</a>,
-  <a href="http://java.sun.com/javase/6/docs/api/java/lang/Comparable.html">Comparable</a>,


Removed this link to Comparable because there is already a link to Comparable#compareTo.

Marcono1234 · 2021-02-19T20:16:32Z

java/ql/src/Violations of Best Practice/Magic Constants/MagicConstantsString.ql

 * Holds if the string is a standard system property as defined in:
 *
- * http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#getProperties()
+ * https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#getProperties()


Did not update this link to Java 11 because the System properties listed for that version might be different than the ones for Java 7.

aschackmull

This is awesome work, thank you very much! Changes LGTM, but pinging docs for a second look @github/docs-content-codeql

felicitymay · 2021-02-23T11:53:48Z

@aschackmull - do you have any thoughts about the failing tests? It looks as is the failure may be caused by one of the experimental queries (UnsafeTlsVersion.ql),but I can't see where this error is coming from:

A fatal error occurred: /home/runner/work/semmle-code/semmle-code/ql/java/ql/src/Advisory/Types/Generics_Common.ql does not exist.

I'm also not sure why we see a Java test failure for JdkInternalAccess.qlref.

In general the docs changes look great 😄
Once we've sorted out these test failures, it might be worth doing testing the changes on staging site for the CodeQL microsite, but otherwise happy for you to merge.

aschackmull · 2021-02-23T12:48:59Z

The qhelp failure is due to the CI checking each .qhelp file as if it was a complete query help. In some cases we share snippets of qhelp between different queries, which are then imported into those queries' qhelp files. We used to name both complete qhelp and qhelp snippets with the same extension, but we've started to use .qhelp.inc for such snippets instead, so Generics_Common.qhelp should be renamed to Generics_Common.qhelp.inc for the test to pass.

The other failure is an internal test case that references the public query. This can't be fixed directly in this PR, so I'll handle that separately.

aschackmull · 2021-02-23T12:51:43Z

@Marcono1234 Could you rename the Generics_Common.qhelp file to Generics_Common.qhelp.inc and update the references to it? (There are a couple in the other qhelp files in the same dir.) I believe that should fix the qhelp preview check.

Marcono1234 · 2021-02-23T22:21:32Z

I have also changed the extension of other help files for which no query file exists, I hope that is alright.
Thought wouldn't it be better to use .inc.qhelp (and adjust CI) so IDEs / editors can still detect the file type and provide syntax highlighting?

aschackmull · 2021-02-24T08:37:45Z

I have also changed the extension of other help files for which no query file exists, I hope that is alright.

That's excellent, thanks!

Thought wouldn't it be better to use .inc.qhelp (and adjust CI) so IDEs / editors can still detect the file type and provide syntax highlighting?

Probably yes. But that's a separate issue.

aschackmull · 2021-02-24T14:34:40Z

Actually, I'll try to fix up our CI such that we can use .inc.qhelp instead - your suggestion gained some quick support.

java/ql/src/Likely Bugs/Serialization/NonSerializableComparator.qhelp

aschackmull · 2021-02-25T09:26:18Z

The CI has been updated to support .inc.qhelp as per your suggestion. Could you rebase to latest main (to include the fixed workflow) and rename the extensions to .inc.qhelp instead? Also, there are a number of the renamed qhelp files that are shared between multiple languages (sharing is done by copying the files and having a script verify that they're kept in sync), so that check currently fails; see config/identical-files.json and config/sync-files.py. The script is normally used to easily sync changes between the copies, but it doesn't support renames, so the copies and their references in the json doc needs to be renamed as well.

Where possible update Java documentation links to Java 11. Additionally update some other links to use HTTPS.

Marcono1234 · 2021-02-26T00:37:57Z

Have rebased onto main, adjusted the extensions to .inc.qhelp and synchronized the necessary changes to the other languages.
For .qhelp renaming for the other languages, I have created #5275.

geoffw0

CPP changes LGTM.

aschackmull · 2021-02-26T12:37:09Z

javascript/ql/src/Comments/CommentedOutCode.qhelp

 <qhelp>
-  <include src="CommentedOutCodeQuery.qhelp" />
+  <include src="CommentedOutCodeQuery.inc.qhelp" />
  <include src="CommentedOutCodeCommon.qhelp" />


This one also needs a rename in this PR, since it's being changed.

It needs a rename because 1) that file is changed and 2) there is no corresponding .ql file, right?
I actually had already updated this by accident (but forgot the file rename), then reverted it and moved it to #5275 🤦

Is there a built script I can run to notice these errors myself or will it only work through trial and error by you having to check the internal CI errors?

Have amended this change to the last commit and force-pushed. There are no other changes so the diff only shows these changes.

It needs a rename because 1) that file is changed and 2) there is no corresponding .ql file, right?

Correct.

Is there a built script I can run to notice these errors myself or will it only work through trial and error by you having to check the internal CI errors?

Unfortunately not. The bulk qhelp processing actually doesn't require these renames at the moment - it is only the CI that chokes on the the .qhelp-without-.ql situation.

Looks like there's a similar situation for javascript/ql/src/Metrics/FLinesOfSimilarCodeCommon.qhelp

Sorry that I missed that. I have force-pushed the changes and went through all changed files to make sure that I have not missed another case, but I did not find any.

aschackmull · 2021-03-03T11:56:03Z

  Error: 3-03 10:43:57] [ERROR] generate query-help> /home/runner/work/semmle-code/semmle-code/ql/python/ql/src/Lexical/CommentedOutCode.qhelp: Could not find include file CommentedOutCodeExample.inc.qhelp
  Error: 3-03 10:43:57] [ERROR] generate query-help> /home/runner/work/semmle-code/semmle-code/ql/python/ql/src/Lexical/FCommentedOutCode.qhelp: Could not find include file CommentedOutCodeExample.inc.qhelp

aschackmull

Everything looks good now, thanks! I'll merge this indirectly to handle the failing test, so as long as you don't change the head of this PR anymore it should show up as merged once I get that sorted.

Java: merge #5226

RasmusWL · 2021-03-04T16:24:38Z

Thanks @Marcono1234 👍 really awesome of you to do this 💪

Marcono1234 requested review from a team, felicitymay and jf205 as code owners February 19, 2021 20:08

github-actions bot added C# documentation Java labels Feb 19, 2021

Marcono1234 commented Feb 19, 2021

View reviewed changes

aschackmull previously approved these changes Feb 23, 2021

View reviewed changes

aschackmull added the no-change-note-required This PR does not need a change note label Feb 23, 2021

Marcono1234 dismissed aschackmull’s stale review via 070bca6 February 23, 2021 22:20

aschackmull reviewed Feb 25, 2021

View reviewed changes

java/ql/src/Likely Bugs/Serialization/NonSerializableComparator.qhelp Outdated Show resolved Hide resolved

Marcono1234 mentioned this pull request Feb 25, 2021

Some .qhelp files appear to be unused #5274

Closed

Marcono1234 added 2 commits February 26, 2021 00:43

Update Java documentation links to Java 11

e21cbe8

Where possible update Java documentation links to Java 11. Additionally update some other links to use HTTPS.

Java: Use .inc.qhelp extension for included help files

53dc2ce

Marcono1234 force-pushed the marcono1234/update-java-documentation-links branch from 070bca6 to 70d8e48 Compare February 25, 2021 23:44

Marcono1234 requested review from a team as code owners February 25, 2021 23:44

github-actions bot added C++ JS Python labels Feb 25, 2021

Marcono1234 force-pushed the marcono1234/update-java-documentation-links branch from 70d8e48 to 4513785 Compare February 26, 2021 00:13

Marcono1234 mentioned this pull request Feb 26, 2021

Use .inc.qhelp extension for included help files #5275

Merged

geoffw0 reviewed Feb 26, 2021

View reviewed changes

aschackmull reviewed Feb 26, 2021

View reviewed changes

Marcono1234 force-pushed the marcono1234/update-java-documentation-links branch 2 times, most recently from 6f70ac5 to 3d56566 Compare March 2, 2021 21:09

Sync .qhelp file renaming to other languages

b9c0193

Marcono1234 force-pushed the marcono1234/update-java-documentation-links branch from 3d56566 to b9c0193 Compare March 3, 2021 14:39

aschackmull mentioned this pull request Mar 4, 2021

Java: merge #5226 #5324

Merged

aschackmull approved these changes Mar 4, 2021

View reviewed changes

felicitymay removed the request for review from jf205 March 4, 2021 11:24

smowton changed the title ~~Update Java documentation links to Java 11~~ Update Java documentation links to Java 11; use .inc infix for included qhelp files Mar 4, 2021

aschackmull added a commit that referenced this pull request Mar 4, 2021

Merge pull request #5324 from aschackmull/java/merge-5226

f9365dc

Java: merge #5226

aschackmull merged commit 45f5228 into github:main Mar 4, 2021

Marcono1234 deleted the marcono1234/update-java-documentation-links branch March 4, 2021 20:10

Marcono1234 mentioned this pull request Jun 14, 2021

Java: Timing attacks while comparing results of cryptographic operations #6006

Merged

Marcono1234 mentioned this pull request Mar 15, 2022

Java: CWE-378: Temp Directory Hijacking Race Condition Vulnerability #4473

Open

		<li>
		Java Language Specification, 3rd ed: <a href="https://docs.oracle.com/javase/specs/jls/se6/html/conversions.html#5.4">5.4 String Conversion</a>.

Update Java documentation links to Java 11; use .inc infix for included qhelp files #5226

Update Java documentation links to Java 11; use .inc infix for included qhelp files #5226

Uh oh!

Conversation

Marcono1234 commented Feb 19, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aschackmull left a comment

Choose a reason for hiding this comment

Uh oh!

felicitymay commented Feb 23, 2021

Uh oh!

aschackmull commented Feb 23, 2021

Uh oh!

aschackmull commented Feb 23, 2021

Uh oh!

Marcono1234 commented Feb 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aschackmull commented Feb 24, 2021

Uh oh!

aschackmull commented Feb 24, 2021

Uh oh!

Uh oh!

aschackmull commented Feb 25, 2021

Uh oh!

Marcono1234 commented Feb 26, 2021

Uh oh!

geoffw0 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Marcono1234 Feb 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aschackmull commented Mar 3, 2021

Uh oh!

aschackmull left a comment

Choose a reason for hiding this comment

Uh oh!

RasmusWL commented Mar 4, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Update Java documentation links to Java 11; use `.inc` infix for included qhelp files #5226

Update Java documentation links to Java 11; use `.inc` infix for included qhelp files #5226

Marcono1234 commented Feb 23, 2021 •

edited

Loading

Marcono1234 Feb 26, 2021 •

edited

Loading