Comments (0)

By blobaugh, July 7, 2010 1:57 pm

With fopen becoming increasingly scarce on web hosts, CURL is becoming a better and better solution to retrieving remote content with PHP. CURL is a fairly standard PHP module and chances are if your host does not support it a simple ticket will have them installing it for you.

Because I am constantly loosing my CURL code and being forced to look it up again I decided to post my favorite CURL function for future reference for myself. I hope that you may get some use out of this as well.

 
/**
 * Get a web file (HTML, XHTML, XML, image, etc.) from a URL.  Return an
 * array containing the HTTP server response header fields and content.
 */
function curlFile($url) {
    $options = array(
        CURLOPT_RETURNTRANSFER => true,     // return web page
        CURLOPT_HEADER         => false,    // don't return headers
        CURLOPT_FOLLOWLOCATION => true,     // follow redirects
        CURLOPT_ENCODING       => "",       // handle all encodings
        CURLOPT_USERAGENT      => "blob curler 1.2", // who am i
        CURLOPT_AUTOREFERER    => true,     // set referer on redirect
        CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
        CURLOPT_TIMEOUT        => 120,      // timeout on response
        CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
    );
 
    $ch      = curl_init( $url );
    curl_setopt_array( $ch, $options );
    $content = curl_exec( $ch );
    $err     = curl_errno( $ch );
    $errmsg  = curl_error( $ch );
    $header  = curl_getinfo( $ch );
    curl_close( $ch );
 
    $header['errno']   = $err;
    $header['errmsg']  = $errmsg;
    $header['content'] = $content;
    return $header;
}
 

Additional Reading:
Reading a Remote File Using PHP
How to get a webpage using CURL

Comments (1)

By blobaugh, June 24, 2010 9:40 am

I am currently working on developing an iPhone app for Northwest Nazarene University. Part of this app will need to connect to the school's webserver to retreive items such as news feeds and shared photo galleries. Being primarily a web developer I have turned to two projects created by Nitobi persons, PhoneGap and jQTouch.

PhoneGap allows me to use my current HTML, CSS, and Javascript knowledge to build a site. Then using the PhoneGap xcode project I simply drop the web files into a directory and PhoneGap transforms them into a glorious iPhone app.

jQTouch is a jQuery based library that makes the app look and feel like a real iPhone app.

This setup worked beautifully until I needed to connect to the school's webserver and retreive content. I began to run into cross site scripting (XSS) headaches. After days of pounding my head into my desk I ran across a neat little feature in Apache that allows you to turn off the XSS protection on the server. You may be thinking that sounds insecure, and you would be right, but if you are trying to pull content off your server with Javascript from remote locations it may be just the thing you need to get it working. This does not come built in Apache, it is a module you have to add in. It was pretty trivial. Here are the directions for setting up mod_headers in Ubuntu

To check and see if mod_headers was actually loaded save the following to a PHP file and view it from the server in your browser

print_r(apache_get_modules());
 

If that was successful mod_headers should be listed. You can now move on to the more fun stuff.

There are two ways in which you can allow XSS. The first is through a .htaccess file. This method works great, however it enables XSS for all files in the directory. The second is through PHP header(). The PHP method allows you to target more specifically what files are XSS capable, and even would allow you to do some other safety checks in your code before setting the XSS header.

.htaccess

Header add Access-Control-Allow-Origin "*"
 

PHP

header('Access-Control-Allow-Origin: *');
 

Another thing to take note of is the *. The * means anyone from anywhere may do XSS. You can however specify specific hosts that have access. See the link at the bottom of this post for more information on that.

Congratulations! You should now have an XSS compatible website. Make extra sure that all of your security is in place because XSS opens up a whole new can of worms for crackers. Ensure all input from users is cleaned before it touches your database or anything else remotely sensitive.

Articles that helped me along the way:
Access-Control-Allow-Origin Multiple Origin Domains?
Configure Apache To Accept Cross-Site XMLHttpRequests on Ubuntu
HTTP Access Control
Server-Side Access Control
Social Javascript (cross-site ajax)

Comments (0)

By blobaugh, June 16, 2010 1:00 pm

When pulling in array from outside data sources you often receive an array back. The problem is that sometimes even though you know you should have an array your application does not and therefore assigns it to the stdObject object, which of course is nothing. To make it usable you must convert it back into an array. With a simple object that may be as simple as a cast, but when you are working with large complex datasets that may be several layers deep you need to make sure you get at all of them. Enter beautiful recursion. After playing with PHPs array_walk_recursive() for a bit I hit on a custom recursive function that does exactly the job. Simply pass it your object and it will munch away at it trying to convert it into a PHP array. Take a gander at the code for this below.

 
function object_to_array($obj) {
    if(is_object($obj)) $obj = (array) $obj;
    if(is_array($obj)) {
        $new = array();
        foreach($obj as $key => $val) {
            $new[$key] = object_to_array($val);
        }
    }
    else $new = $obj;
    return $new;
}
 

Comments (0)

By blobaugh, June 16, 2010 9:36 am

It is quite common to run into situations where PHP errors are suppressed or written to a log file somewhere on a live, non-development server. If you are troubleshooting and need to see the error that can be a pain. Luckily PHP and Apache have provided us with two great workarounds. The first happens through a PHP function call in a script, the second is a .htaccess directive which does require special configuration of Apache, but the majority of servers I have worked with allow this functionality.

Place this code near the top of a PHP file

 
ini_set("display_errors","1");
error_reporting(E_ALL);
 

This code goes into the .htaccess file. Just be sure your .htaccess file is in the proper directory relative to the code you are trying to debug

 
php_value display_errors on
 

Just be sure you turn error printing back off when you are done.

Comments (0)

By blobaugh, June 14, 2010 1:38 pm

I am in a situation where I need analytics data and none is provided. The entire 35k+ file website is all static HTML running on a Windows IIS server. I am not familiar enough with Windows Server to know what sort of scripting capabilities there are, but I do have PHP available. I wrote a simple script that uses the PHP 5 RecursiveDirectorIterator to look through the entire site and append the Google Analytics code to the closing body tag. It then rewrites the file it is looking at.

If you are stuck in a similar situation where you do not have scripting abilities you may want to consider giving this script a go, or altering it to fit whatever language you are using. Works like a charm.

 
$directory = 'where your files are';
 
$it = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($directory));
 
while($it->valid()) {
 
    if (!$it->isDot()) {
        $c = file_get_contents($it->key());
        $c = str_replace('</body>', YOUR ANALYTICS CODE HERE . '</body>', $c);
        $fh = fopen($it->key(), 'w');
        fwrite($fh, $c);
        fclose($fh);
        echo "Replaced body in: " . $it->key();
    }
 
    $it->next();
}
 

Might be worth it to ensure you are looking only at .html and .htm files if you have a complex set. Additionally, if you have a very large set of files you may need to look at timeout issues.

Comments (0)

By blobaugh, June 3, 2010 7:44 pm

Quite often HTTPS is required for security reasons, but the question is how to force HTTPS with PHP?  Really it is quite simple. Copy the following function into your functions file and call it on top of every page that requires HTTPS to ensure it is used.

 
function forceHttps() {
 $pageURL = 'http';
 if (!isset($_SERVER["HTTPS"]) || $_SERVER["HTTPS"] != "on") {
	header("Location: https://" . $_SERVER["SERVER_NAME"].$_SERVER["REQUEST_URI"]);
 }
}
 

Comments (0)

By blobaugh, June 3, 2010 7:37 pm

I've run across a couple instances where I needed to either remove empty array elements or perform the equivalent of a trim() on PHP arrays. There are two pretty good ways to do this.

The first way involves building a custom function to loop through an array and unset empty elements like so:

function arrayRemoveEmpty($Array) {
	foreach($Array AS $k => $v) {
		if($Array[$k] == '') {
			unset($Array[$k]);
		}
	}
	return $Array;
}
 

The second is probably a more preferred, and certainly faster way. PHP has a built in function to handle filtering out empty elements, and it can be extended to do virtually anything you can imagine. Take a look:

$Array = array_filter($Array);
 

Comments (0)

By blobaugh, May 17, 2010 6:25 am

Ever wonder how many pages, words or tt_content element you have in your Typo3 install? I am working with two large Typo3 sites and I was curious so I wrote a little piece of code. It works really well and is super quick. Here is what the output looks like:

Counting Pages
SELECT COUNT(uid) AS num_pages FROM `pages`Counting Pages Not Deleted
SELECT COUNT(uid) AS num_pages FROM `pages` WHERE deleted='0'

Counting Content
SELECT COUNT(uid) AS num_content FROM `tt_content`

Counting Words
SELECT bodytext FROM `tt_content`

Totals
Num Pages: 4505
Num Pages Not Deleted: 2948
Num Content: 5152
Num Words: 28,394,588

If you would like to implement this yourself I provided the code below, instead of reinventing the wheel yourself.

(I put mine in the typo3conf folder for ease of use)

 
<?php
/* gather the db connection info from typo3 */
require_once('localconf.php');
error_reporting(E_ALL);
$mysqli = new mysqli($typo_db_host, $typo_db_username, $typo_db_password, $typo_db);
 
/* check connection */
if (mysqli_connect_errno()) {
    printf("Connect failed: %s\n", mysqli_connect_error());
    exit();
}
 
/* db queries */
$sql_count_pages = "SELECT COUNT(uid) AS num_pages FROM `pages`";
$sql_count_pages_not_deleted = "SELECT COUNT(uid) AS num_pages FROM `pages` WHERE deleted='0'";
$sql_count_content = "SELECT COUNT(uid) AS num_content FROM `tt_content`";
$sql_bodytext = "SELECT bodytext FROM `tt_content`";
 
/* useful vars */
$num_pages = 0;
$num_pages_not_deleted = 0;
$num_content = 0;
$num_words = 0;
 
echo '<b>Counting Pages</b>';
echo '<br/>'.$sql_count_pages;
$result = $mysqli->query($sql_count_pages);
$result = $result->fetch_assoc();
$num_pages = $result['num_pages'];
 
echo '<b>Counting Pages Not Deleted</b>';
echo '<br/>'.$sql_count_pages_not_deleted;
$result = $mysqli->query($sql_count_pages_not_deleted);
$result = $result->fetch_assoc();
$num_pages_not_deleted = $result['num_pages'];
 
echo '<br/><br/><b>Counting Content</b>';
echo '<br/>'.$sql_count_content;
$result = $mysqli->query($sql_count_content);
$result = $result->fetch_assoc();
$num_content = $result['num_content'];
 
echo '<br/><br/><b>Counting Words</b>';
echo '<br/>'.$sql_bodytext;
$result = $mysqli->query($sql_bodytext);
while($r = $result->fetch_assoc()) {
	$num_words += str_word_count($r['bodytext']);
}
 
echo '<br/><br/><br/><b>Totals</b>';
echo '<br/>Num Pages: ' . $num_pages;
echo '<br/>Num Pages Not Deleted: ' . $num_pages_not_deleted;
echo '<br/>Num Content: ' . $num_content;
echo '<br/>Num Words: ' . number_format($num_words, 0, '.', ',');
 
$mysqli->close();
?>
 

Comments (1)

By blobaugh, April 27, 2010 10:43 am

Last week I posted a query that will return the number of words in a column of a table in a MySQL database. I was looking through a huge database and the query took a really really long time. Today I decided to rewrite it in PHP and see if it was faster. Much Much Much faster. PHP has a built in function called str_word_count that is super quick at counting. If you have a huge database and access to PHP I definitely recommend this method over the query I  had before.

 
/* gather the db connection info */
$db_host = 'localhost';
$db_username = '';
$db_password = '';
$db = '';
 
$mysqli = new mysqli($db_host, $db_username, $db_password, $db);
 
/* check connection */
if (mysqli_connect_errno()) {
    printf("Connect failed: %s\n", mysqli_connect_error());
    exit();
}
 
/* db queries */
$sql_text = "SELECT text_to_look_through FROM `table_containing_text`";
 
/* useful vars */
$num_words = 0;
 
echo '<br/><br/><b>Counting Words</b>';
echo '<br/>'.$sql_text;
$result = $mysqli->query($sql_text);
while($r = $result->fetch_assoc()) {
	$num_words += str_word_count($r['bodytext']);
}
 
echo '<br/><br/><br/><b>Totals</b>';
echo '<br/>Num Words: ' . number_format($num_words, 0, '.', ',');
 
$mysqli->close();
 

Comments (0)

By blobaugh, March 10, 2010 7:44 pm

Zac Vineyard wrote a great tutorial on building a basic Authorize.net payment for and connecting to their payment gateway. Zac used some of my code and was kind enough to reference me for it, and since he wrote such a clean tutorial I see no need to write my own (used my code anywho!).

Check his out!

Build a Basic Authorize.net Payment Form