Return response-details for failed http-task

[iceman91176]

We just started with conductor-oss and while playing around with error-handling scenarios for the http-task we discovered that the response for failed http-tasks is pretty "thin".

It just returns the underlying java exception, which is not very useful

conductor/http-task/src/main/java/com/netflix/conductor/tasks/http/HttpTask.java

Lines 184 to 190 in d4c9bbf

	} catch (RestClientException ex) {
	LOGGER.error(
	String.format(
	"Got unexpected http response - uri: %s, vipAddress: %s",
	input.getUri(), input.getVipAddress()),
	ex);
	String reason = ex.getLocalizedMessage();

Is there a reason for this behaviour ? Or was the rjust no need for it ?

IMHO returning the statuscode, a body if it present, headers,etc. would make it much easier to react to various response-codes.

This can easily be achieved by catching a RestClientResponseException instead or in addition of catching the RestClientException. This exception allows to get response-statuscode, body etc.

I wrote a quick implentatio in our dev-environment and it works like a charm.

Instead of returning

{
  "response": "java.lang.Exception: 404 Not Found: \"{}\""
}

we now get

{
  "response": {
    "headers": {
      "Date": [
        "Thu, 13 Feb 2025 15:11:37 GMT"
      ],
      "Content-Type": [
        "application/json; charset=utf-8"
      ],
      "Content-Length": [
        "2"
      ],
      "Connection": [
        "keep-alive"
      ],
      "Report-To": [
        "{\"group\":\"heroku-nel\",\"max_age\":3600,\"endpoints\":[{\"url\":\"https://nel.heroku.com/reports?ts=1739454379&sid=e11707d5-02a7-43ef-b45e-2cf4d2036f7d&s=vUANf4oyOnniVPH6aMzDqrkITV2bAsW1C06BlZKpZ50%3D\"}]}"
      ],
      "Reporting-Endpoints": [
        "heroku-nel=https://nel.heroku.com/reports?ts=1739454379&sid=e11707d5-02a7-43ef-b45e-2cf4d2036f7d&s=vUANf4oyOnniVPH6aMzDqrkITV2bAsW1C06BlZKpZ50%3D"
      ],
      "Nel": [
        "{\"report_to\":\"heroku-nel\",\"max_age\":3600,\"success_fraction\":0.005,\"failure_fraction\":0.05,\"response_headers\":[\"Via\"]}"
      ],
      "X-Powered-By": [
        "Express"
      ],
      "X-Ratelimit-Limit": [
        "1000"
      ],
      "X-Ratelimit-Remaining": [
        "999"
      ],
      "X-Ratelimit-Reset": [
        "1739454436"
      ],
      "Vary": [
        "Origin, Accept-Encoding"
      ],
      "Access-Control-Allow-Credentials": [
        "true"
      ],
      "Cache-Control": [
        "max-age=43200"
      ],
      "Pragma": [
        "no-cache"
      ],
      "Expires": [
        "-1"
      ],
      "X-Content-Type-Options": [
        "nosniff"
      ],
      "Etag": [
        "W/\"2-vyGp6PvFo4RvsFtPoIWeCReyIC8\""
      ],
      "Via": [
        "1.1 vegur"
      ],
      "CF-Cache-Status": [
        "HIT"
      ],
      "Age": [
        "5118"
      ],
      "Server": [
        "cloudflare"
      ],
      "CF-RAY": [
        "9115c0823dd4e85c-FRA"
      ],
      "alt-svc": [
        "h3=\":443\"; ma=86400"
      ],
      "server-timing": [
        "cfL4;desc=\"?proto=TCP&rtt=7088&min_rtt=6774&rtt_var=3169&sent=3&recv=6&lost=0&retrans=0&sent_bytes=219&recv_bytes=1088&delivery_rate=157158&cwnd=251&unsent_bytes=0&cid=2eb3526b5401d09e&ts=20&x=0\""
      ]
    },
    "reasonPhrase": "Not Found",
    "body": null,
    "statusCode": 404
  }
}

I could provide a PR if there is interest for this "feature"

[garfur42]

I was precisely looking into the same thing when I stumbled across this post. I would like to replace applications which "glues" together rest/web request flows and some of them need minor logic in order to do alternatives depending on error responses.

[hpapin]

It could be very useful for me !

[mikebski]

Yeah, we should be able to do something like never throw an exception when an HTTP returns non-2xx or something.

Could also have a configuration param for the http method that gives expected HTTP codes so you could say [200, 404] that would not throw an error and just set the response code and return an empty body.

This would allow the pattern of a switch after an HTTP call for 200 -> do nothing and 404 -> create resource so that we could easily create idempotent requests.

[mikebski]

I submitted a PR here: #544

[jeffbulltech]

Hey @mikebski sorry for the late reply! I added PR 544 to our roadmap and sent it to our engineering team for triage.