Detecting Character Encoding In Coldfusion

By Josh Highland in coldfusion

Most of the time I only work with UTF-8 or UTF-16 character encodings. With that said, recently I was working on an old coldfusion project. Do to database restrictions, required all inputs from the user into the application to be Latin-1 (ISO-8859-1) character encoded.

Coldfusion does not have a good way of checking the character encoding of a string, so I had to pull out some of my Java skillz.

Dropping into Java from ColdFusion is a lot easier then it sounds. Check it out

   
   //use this to test character encoding. We only want Latin-1
    encoder = createObject("java", "java.nio.charset.Charset").forName("ISO-8859-1").newEncoder();

    if(encoder.canEncode(testVar))
    {
        //its Latin-1
    }
    else
    {
        /NOT Latin-1
    }

The Java layer of Coldfusion is very powerful and under utilized by many developers.

If you want to see some true ColdFusion magic, check out the ColdBox Framework, http://ColdBoxFramework.com

PHP and cURL in parallel

By Josh Highland in php

I was recently working on a php based project for a client. I needed to pull in content from 10 different URLs, then process that data into something useful.

I have a growing love affair with cURL, so it was my natural selection for the project.

In my first approach, I looped over the list of URLs, sent the cURL request, wait for the response, stored the data. repeat until done.

That worked, but it was very slow process. The page was taking between 30 – 45 seconds to load. That doesn’t sound long, but in terms of web applications, that’s an eternity.

I tried several tricks to speed the application up, but the bottle neck was the cURL calls. Each call was done in a synchronous manner. Every call to cURL had to be completed before the next could be made.

After doing to research on how to speed this up, I came across the following php cURL functions:

curl_multi_init();
curl_multi_add_handle();
curl_multi_select();
curl_multi_exec();
curl_multi_getcontent();
curl_multi_info_read();
curl_multi_remove_handle();

Using these methods allow for cURL to send asynchronous requests, solving my pervious problem.

A little more searching, and I found an awesome wrapper library that takes the guesswork out of the using the curl_multi methods. “ParallelCurl” https://github.com/petewarden/ParallelCurl

The sample code provided with the library was very straight forward and super easy to use.

After using asynchronous cURL calls via the ParallelCurl library, I was able to reduce the page load time from 45 seconds to 15 seconds. It’s still slow, but it’s a HUGE improvement, and it makes the application usable, and reduces load on my server. It’s a win-win-win situation!

Using cURL and PHP to upload files through a form post

By Josh Highland in php

Lately I have been working on a project that requires me to use PHP to interact with a REST based service. cURL is the logical choice for making HTTP calls to the REST service.

I love cURL, I’ve blogged about it before, but I recently ran into some major issues.

The REST service I was using required me to send two files along with some meta information. easy enough. I used the following code:

$postFields = array();

//files
$postFields['file'] = "@$filePath";
$postFields['thumbnail'] = "@$thumbnailPath";

//metaData
$postFields['title'] = "$title";
$postFields['description'] = "$description";
$postFields['tags'] = "$tags";
$postFields['licenseinfo'] = "$licenseinfo";
$postFields['token'] = "$userToken";

$curl_handle = curl_init();

curl_setopt($curl_handle, CURLOPT_URL, $api_url);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl_handle, CURLOPT_POST, true);
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, $http_post_fields);

//execute the API Call
$returned_data = curl_exec($curl_handle);

The code worked great, sending an “at” sign (@) before the file path makes sure that cURL sends the file as part of a “multipart/form-data” post. Exactly what we needed.

The form post from cURL worked great, but the REST service was retuning a 400 error and saying “The specified thumbnail file is not supported.”. I was at a loss. The service documentation stated the “jpg, jpeg, gif, and png” files were supported.

I ended up contacting the developers of the service who told me that the content type for the file had to be set to “image/jpg” (for jpg).

After pouring through the cURL documentation and not finding anything about how to set the content type for a single file in a “multipart/form-data” post, I turned to Goolge. My searches with about as helpful as the cURL docs. I sent a few hours hacking my code and trying some things, I ever read some posts from 2008 saying that is was not possible to do. Then, I got a break through, a single ray of light. On a message board was a single sentence replay. “You should try this… $image;type=image/jpg”.

That was the break through I needed. Below is final updated code:

$postFields = array(); 

//files
 $postFields['file'] = "@$filePath";

//get the extension of the image file
$tumbnailExtention = preg_replace('/^.*\.([^.]+)$/D', '$1', $thumbnailPath);
$postFields['thumbnail'] = "@$thumbnailPath;type=image/$tumbnailExtention";

//metaData
$postFields['title'] = "$title";
$postFields['description'] = "$description";
$postFields['tags'] = "$tags";
$postFields['licenseinfo'] = "$licenseinfo";
$postFields['token'] = "$userToken"; 

$curl_handle = curl_init();

curl_setopt($curl_handle, CURLOPT_URL, $api_url);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl_handle, CURLOPT_POST, true);
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, $http_post_fields); 

//execute the API Call
$returned_data = curl_exec($curl_handle);

In summary, if you need to set the content type of a file being sent an image through cURL, via a POST, use the following format:
$postFields[‘file’] = “@PATHTOFILE;type=CONTENTTYPEHERE”;