TL;DR Java 1.9 does not like reflection as much as previous versions and access to proprietary packages is not possible out of the box.
Java 1.9 is out, and many companies now will be evaluating Java 1.9 and migrating their code base over to Java 1.9. I am now going through a similar process.
I have some fairly simple code that I was using for HTTP requests. This was originally written in Java 1.8 and to keep my codebase as simple as possible and I wanted to use no external libraries, therefore for HTTP requests I used an HttpURLConnection.
This started to fail on Java 1.9.
When I looked in my code more carefully I realised that it wasn’t HttpURLConnection that was failing in Java 1.9 it was my use of Reflection to bypass some constraints of the HttpUrlConnection that was failing.
What constraints?
With the HttpUrlConnection I could not find a way to get the actual request headers being sent.
I could see that the URLConnection class does have all the request headers:
private MessageHeader requests;
But it keeps them all to itself.
The class does support a mechanism to return these with the getRequestProperties method, which is public.
public Map<String, List<String>> getRequestProperties() {
Huzzah!
But, wait a minute.
public Map<String, List<String>> getRequestProperties() {
this.checkConnected();
return this.requests == null?Collections.emptyMap():this.requests.getHeaders((String[])null);
}
getRequestProperties does not actually want to return them if we are connected, the checkConnected method throws an exception if we try.
Fair enough.
I debug the code and find out when we can get them, and I’ll take them after they are set, but before we are connected.
But I could not actually find that point.
int statusCode = con.getResponseCode();
Prior to the above line I am not connected, and there are no request headers accessible.
After the above line, the request has been made, I am connected and the headers are not accessible.
I tried disconnecting, and a whole bunch of stuff, but since I had a problem to solve and the ‘class’ was getting in my way, I turned to Reflection to solve it.
What I did that failed
Since I new that the requests field on the URLConnection had the information that I needed, I simply used reflection to go and get the information that I needed.
Field field = con.getClass().getDeclaredField("requests");
field.setAccessible(true);
MessageHeader requestHeaders = (MessageHeader) field.get((URLConnection) con);
System.out.println(requestHeaders.getHeaders().size());
lastRequest = new HttpRequestDetails();
int keyIndex = 0;
while(keyIndex < requestHeaders.getHeaders().size()){
lastRequest.addHeader(requestHeaders.getKey(keyIndex), requestHeaders.getValue(keyIndex));
keyIndex++;
}
And all of that was very jolly, until Java 1.9 came along.
Migrating to Java 1.9
After debugging the problem I faced, I found the following blog post from codefx.org provided useful information:
The problem I faced was that when I migrated to Java 1.9 I faced the issue of:
Error:(7, 19) java: package sun.net.www does not exist
Error:(169, 13) java: cannot find symbol
symbol: class MessageHeader
location: class com.javafortesters.course.casestudy.http.HttpRequestSender
Error:(169, 45) java: cannot find symbol
symbol: class MessageHeader
location: class com.javafortesters.course.casestudy.http.HttpRequestSender
Now I found this odd.
The import of sun.net.www.MessageHeader wasn’t throwing any errors. Until compilation.
import sun.net.www.MessageHeader;
And when I looked in the code for URLConnection the responses field was still there and was still of of type sun.net.www.MessageHeader
So the package does exist, but just not for me.
Java 1.9 has finally made proprietary packages inaccessible. This was a long time warning in Java and has now come to fulfillment.
It is still in the JRE, but not accessible to me due to Java 1.9 changes.
Very well then.
Since the getRequestProperties method still exists, I decided to bypass the check for this.checkConnected();
I’m sure the library probably has very good reasons for not wanting me to access the List while we are connected, but in my code I view this as low risk since I have a single thread, I’ve made the request, I just want to document the headers that were sent.
So I used reflection to toggle the connectedfield value so that the checkConnected() method would not throw an exception.
Field connected = con.getClass().getSuperclass().getSuperclass().getDeclaredField("connected");
connected.setAccessible(true);
connected.setBoolean((URLConnection) con, false);
Map<String, List<String>> requestHeaders = con.getRequestProperties();
for(String header : requestHeaders.keySet()){
StringBuilder headerValue = new StringBuilder();
for(String headerText : requestHeaders.get(header)){
headerValue.append(headerText);
}
lastRequest.addHeader(header, headerValue.toString());
}
connected.setBoolean((URLConnection) con, true);
connected.setAccessible(false);
And this worked in Java 1.9, and 1.8 and 1.7
But… Java 1.9 Doesn’t like reflection as much
But Java 1.9 tells me that this reflective access operation is ILLEGAL!
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.javafortesters.course.casestudy.http.HttpRequestSender
(file:/) to field java.net.URLConnection.connected
WARNING: Please consider reporting this to the maintainers of com.javafortesters.course.casestudy.http.HttpRequestSender
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
I assume this is due to the new Module system in Java 1.9
So while my code works, it will probably fail again in the near future.
Reflection Comes with Risk
Reflection always comes with the Risk that the underlying code you are using changes.
And it has every right to do so.
I should not be accessing its private fields and methods.
But sometimes, when testing, I need access to features and functionality that, for seemingly arbitrary reasons, the class does not support.
Had this library been written to fully support a ’testing’ process, where we don’t just want to make requests, we want to know exactly what details were sent in that request, then I wouldn’t have this problem.
The reason I learned to use reflection in Java in the first place was because I’ve worked with ‘frameworks’ that do stuff, that I want to use as part of my test process to validate results, or to trigger situations that are necessary for testing, but which the ‘framework’ decided that no-one would want to do that.
In theory, there is nothing wrong with the URLConnection telling me what the request headers are. It could have added them to a Synchronised or immutable collection, it is pure documentation.
Sometimes good solid technical reasons for functionality gets in the way of testing.
Alternatives
I could have added another HTTP library to my code to make the requests and give me access to the raw data.
I could have added a code based HTTP Proxy, sent all requests through the proxy and used the proxy to gain access to the headers.
All of that was possible, but went against my strategic aims of keeping the code base as free of external dependencies as possible, so I adopted a tactical maneuver to gain access to what I needed.
Final Results
In the public release of the above code I have removed all the reflection. But I have kept it in the code I use for internal testing because I can control the JVM and settings that the code runs against.
I will continue to use reflection to bypass constraints in libraries and frameworks that I use as a tactical measure to get work done.
Relying on this approach for strategic work is a risk that might bite you, at the time when you are least flexible to respond, so care needs to be taken when you do this.
