Author: LizzyFin

MySQL LOAD FILE Statement Rescues Huge Database File

A client needed to use an existing database, which turned out to be a big gargantuan thing, but to be able work with it on the Web I needed to convert it from MS Access to MySQL.

Instead of trying to convert the Access files to MySQL I chose to work with delimited text files that were available. Ten text files were used to represent ten tables for the database. It was easy enough to read in the delimited text files into OpenOffice Calc by identifying the delimiting characters and importing the new .csv file into MySQL, but one file in particular was really big and presented a problem.

I tried to work with the big text file in Notepad and HTML-Kit, but it was too cumbersome due to the file size – over 500,000 records! Not enough copy and paste operations could be done in a day to make me want to go through it again when the original database gets updated and I have to convert the scary big text file into MySQL format again. Knowing that the original database can be updated at any time, I needed a way to get that huge file into MySQL with a minimum of effort.

After looking around a bit, I downloaded a piece of conversion software called Full Convert from SpectralCore.

Installation went smoothly and I noted there was an uninstall feature in case the software wasn’t a good fit for my needs. It seemed promising in that there are several databases that can be converted back and forth, but the function to convert a text file to MySQL tables did not work for me. The text file used the characters ‘^’ to terminate fields and ‘~’ to enclose fields. However, the software did not give a choice for the field-enclosing character.

Since it didn’t work for me I uninstalled it, but the Full Convert software may be promising if you’re moving files from one database system to another. Here’s the list of DBs they work with:

Supported source databases
– Microsoft Access, dBase, FoxPro, Microsoft Excel documents, Interbase / Firebird, Lotus 1-2-3, MySQL, Oracle, PostgreSQL, ODBC sources, Paradox, SQL Server, Delimited text files, XML

Supported target databases
– MySQL, SQL Server, Oracle, PostgreSQL, Access, Interbase/Firebird

Further digging lead me to a solution for this problem of not being able to import a really big, delimited text file into a table in MySQL.

LOAD DATA INFILE

The solution is to read the file directly into MySQL using the MySQL statement “LOAD DATA INFILE”.

For my situation the code that worked was this –

LOAD DATA INFILE ‘data.txt’ INTO TABLE databasename.tablename FIELDS TERMINATED BY ‘^’ ENCLOSED BY ‘~’;

Don’t forget the trailing semi-colon and don’t use single quotes around the database and table names (and leave the semi-colon in the space provided for the Delimiter).

How did I find my solution? Searching at WebmasterWorld for “convert text to mysql” did the trick.

A simple post by a fellow developer that was written to help someone else out a couple years ago helped lead me to the LOAD_DATA function. Thanks, dreamcatcher!

Calendar Pick-A-Date Scripts Reviewed

This week I needed to make a form where the user needed to enter two dates in order to produce a report with the selected start and end dates. What were my options?

I considered writing my own php script where I would have separate pull-down boxes for the day, month and year to assure that there was no mistake as to the actual dates. That would take a little extra format massaging to create the date in MySQL format, so I looked for scripts that others had already written. Who wouldn’t?

Formatting dates can drive a person crazy. Does ’06-08-2000′ mean June 8th or the 6th of August? Writing for the Web should be universal. The month-day format that we tend to use in the U.S. is confusing to everybody else who uses the day-month format, and vice-versa.

Writing by hand I like to use this format ’12-Mar-2000′ so there is no ambiguity, but that format needs to be converted to ‘YYYY-MM-DD’ format when sending dates to MySQL.

So, what are my requirements for such a Date-Picker Script?

  • easy, intuitive for the user
  • easy for me with output in MySQL format (YYYY-MM-DD)
  • unrestricted license would be nice

I tried a few different date picker scripts and decided upon SpiffyCalendar because it’s the most advanced, yet easily modified to suit my purposes. Also, there is an established support system via a Yahoo! user group. That’s really nice, and unexpected.

Here’s the scripts I tried and the pros and cons for each…most weren’t as user-friendly as SpiffyCalendar.

 

PHP Calendar

    PROS

  • I like the look of this calendar, but it’s difficult to determine how to modify the calendar other than for css of calendar appearance.
  • Date output is already in MySQL format: YYYY-MM-DD
    CONS

  • Can’t select year from calendar unless you click back or forward to the actual month.
  • Ugly and excessively long query strings in links.

DatePick

    PROS

  • I like the little calendar icon.
    CONS

  • Can’t select year from calendar unless you click back or forward to the actual month.
  • Timestamp is included. Don’t need it.
  • Relies on javascript and pop-up window to select date.
  • Calendar doesn’t update well when moving from month to month. Part of the calendar doesn’t get loaded in so all days are not selectable. Have to manually widen window to see all days.
  • Can’t select year directly, have to click through months to get to next year.
  • No directions on how to modify output.

DateTimePick

    PROS

  • I like the little calendar icon.
  • Can modify output via provided directions.
    CONS

  • Can’t select year from calendar unless you click back or forward to the actual month.
  • Relies on javascript and pop-up window to select date.
  • Calendar doesn’t update well when moving from month to month. Part of the calendar doesn’t get loaded in so all days are not selectable. Have to manually widen window to see all days.
  • Can select year directly, but calendar doesn’t update well.

SpiffyCalendar

Sign up to the Yahoo! Group to download the script. Script used to be offered at the now defunct GeoCities. The above link is the first place I found it posted afterwards.

    PROS

  • Calendar pops into page, not a separate window.
  • Can modify output via provided directions.
  • Can get help at Yahoo! Groups.
  • Unrestricted license.
  • Can refresh page containing calendars as link is not modified after picking a date or two.
  • Highlights holidays, although will have to remove Dec 1st developer’s birthday. (OK, modified holiday array in script on lines 86-87.)
    CONS

  • Format of date will have to be modified for MySQL. (OK, Added "cal1.dateFormat="yyyy-MM-dd"; after instantiation in header for calendar named ‘cal1’.)
  • Relies on javascript.
  • Had to modify the images folder location in the .js file to an absolute address, although things worked fine on my local server with a relative address.

I’m sure there are plenty of these date-picker scripts floating around the Web. Do you have a different one to share with us? Leave a comment!

Review of Popular WordPress Plugins: WP-Cumulus

If you display tags or categories in a sidebar of your WordPress blog, you might like to take a look at the WP-Cumulus plugin. It gets 5 stars by all the reviewers so far and definitely hits the top on the cool factor.

WP-Cumulus displays your tags and/or categories but in a visually-striking way. A rotating cloud of words is presented instead of a plain old list.

Here’s an example of WP-Cumulus in action:

[WP-CUMULUS]

Move your cursor over the display and the rotation will change direction and speed. Click on a word once it has a box around it to get a listing of posts containing that tag or filed in that category. Pretty cool, eh?

The only caution here is that you must be running WP 2.3 or later. WP-Cumulus will not work with earlier versions.

Installation follows the typical plugin format. Simply download the plugin, unpack it, and FTP the plugin folder, called wp-cumulus, to your wp-content/plugins directory.

There are three ways to display your WP-Cumulus cloud:

  • In a page or post like above, using the key [WP-CUMULUS]. Features can be modified under Settings/WP-Cumulus.
  • In your theme anywhere that you insert <?php wp_cumulus_insert(); ?>, probably in a sidebar. Just make sure the sidebar is big enough to display the cloud nicely.
  • As a widget, via ‘Design’->’Widgets’. Edit the widget to modify the size of the cloud and the colors used for the text and background.

Really nice, don’t you think?

Update WordPress to the Latest Version to Secure Your Blog

Aren’t you running the latest version of WordPress? Right now, the WP developers are working on version 2.7 and the latest available non-beta version is 2.6.3.

If you’re not running the latest version of WordPress, why not? There’s no reason not to – in my mind at least. You and your data will be safer when you run the most up-to-date version.

Actually, I did think of one exception and that would be if you use a must-have plugin that isn’t yet updated to run with the latest WordPress, you might want to wait it out for the updated plugin to arrive before updating to the latest WP. I wouldn’t wait too long because it is known that older versions of WP are targets for the dolts out there who are trying to separate you from your money and identity. Check the plugin’s FYI box for version compatibility.

Sometimes your installation of WP won’t be affected by the security or other issues solved in the next release and that depends on the features of WP that you rely upon. For instance, updating from 2.6.2 to 2.6.3 wasn’t necessary for my blogs because I wasn’t using the feature that had a security issue fixed. Not updating to 2.6.3 wasn’t a security risk, so I could put that off because I know another version is on its way shortly.

You can get all the info on the latest updates by scrolling down into your WP Dashboard and reading the WP news links.

Besides being assured that you have the latest and most secure version of WordPress, you’ll get all the tweaks for a smooth ride.

Updating to the next version of WP isn’t that bad. Once you run through it a couple of times updating your software is pretty darn easy.

Here’s the quick steps to a better WordPress experience:

  • Backup your database. You probably should be doing this anyway, right? Did you know that a plugin will automate backing up your WP site? One click backup – how easy can you get?
  • Backup your other pages, those not produced by WordPress, by FTPing a copy somewhere. You do have a place for extra storage, don’t you? If not, try Mozy Online Storage. They’ll give you a few free GB to store anything you like so you don’t glum up your local hard drive.
  • Deactivate all plugins. With the latest versions of WP this is a one-click step.
  • Verify that the first three steps were done successfully.
  • Download the latest WordPress software and unpack the zip file.
  • Delete the old WP pages, EXCEPT wp-config.php, the wp-content folder, the wp-images folder, your .htaccess and robot.txt files.
  • Upload the new WP files. Check if there are plugins or themes that you might use with newer dates in the new wp-content folder. Upload any newer files.
  • Run the upgrade program by visiting http://yourdomain.com/wp-admin/upgrade.php, and clicking on Upgrade.
  • Sign-in to your WP Dashboard and verify the permalinks are set to your liking.
  • Activate plugins, as needed.
  • Feel satisfied that you’re now safely running the latest and greatest WordPress blogging software.

It’s not difficult to run these updates, and you should be backing up your great content anyway, right? To stay on top of the most recent changes in WP, check out the WP news items in your Dashboard.