The Linux Page

POSTing multiple files over HTTPS

I've been trying to send a POST to Apache 2.x using cURL. In itself, that's very easy to do. However, I run modsecurity and when cURL sends a POST that's too large, it actually decides to break the transfer down using an Expect: 100-continue header. That in itself sound good.

Some people said that you could override the Expect by adding the curl option to add a header like this:

  curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));

This sounds all nice, however, it only removes the header from the request, it does not prevent the errors with modsecurity. Not only that, the curl header does not include a size for the files + parameters and that too makes modsecurity go bananas.

I'd bet that without modsecurity it would work, but I just don't have the time to test that.

Instead I created the request "by hand" and used fsockopen(). That works perfectly and sends the expected POST. (note that my POST is similar to an HTTP form which apparently curl has a problem with.)

Creating the request by hand takes a little more knowledge of the HTTP requests and how to compute different things such as the Content-Length, but at least it works.

The errors returned were many, 501, 400 and 500 for most (I also managed to generate a 408.) I have not found a way to fix my PHP to fix the problem. Plus, if my code is to work on customers' machines, then I cannot just tweak (fix) the code on my server...

A few extra notes:

  • There is a limit that is used to determine whether one more or the other should be supported. This limit may be as small as 1Kb and as high as 65Kb. So if your post is 1024 bytes or more, you may start experiencing this problem.
  • Overriding the header doesn't prevent the behavior.
  • The behavior is supposed to only occur when the destination requires authentication (so as to avoid sending large packets to then discover that the user cannot authenticate.)
  • This happens whether you post file attachments or not, it has nothing to do with that.