My Christmas Goodies

from the two best kids in the world.

See and download the full gallery on posterous

Posted via email from The Soistmann Family

Posting del.icio.us Links to WordPress: Finishing Up

On Wednesday, I posted more info about how to clean up my weekly del.icio.us links. There are a few things I’d like to do before I wrap this up.

  1. change all tags and attributes to lowercase
  2. close every dt element
  3. close every dd element
  4. make things a bit more automatic

If we take a closer look at the code for each entry we will see a pattern.


One line has a <DL> followed by the anchor. The next line has a <DD> followed by my comments.

<DT><A HREF="url" LAST_VISIT="1238086010" ADD_DATE="1238086010" TAGS="tagone,tagtwo">Link text</A>
<DD>comments

The only thing that makes this tricky at all is that sometimes the comments span more than one line. We can get around this fairly easily though. All we need to do is put the closing </dd> before all the <DT> tags except the first one. Let’s make that easier by changing the first one to lowercase. We’ll change part of what we did yesterday to accomplish this. Instead of replacing

<DL><p>

with

<dl>

we will replace

<DL><p><DT><A HREF=

with

<dl><dt><a href=

The rest is of the cleanup is pretty straightforward.


Replace

<DT><A HREF="

with

</dd><dt><a href="

and

</A>

with

</a></dt>

and then

LAST_VISIT=[^<]*TAGS="

with

tags="

since I don’t need two of those attributes anyway.

And I almost forgot

<DD>

with

<dd>

Wrap it all up and we have

grep '^> ' < links.diff |awk '{sub(/<DL><p><DT><A HREF=/,"<dl><dt><a href=")};{sub(/<\/A>/,"</a></dt>")};{sub(/<DT><A HREF=/,"</dd><dt><a href=")};{sub(/<DD>/,"<dd>")}{sub(/LAST_VISIT[^<]*TAGS=/,"tags=")};{sub(/^> /, "")};!/<\/DL>/{print}' > foo.html;echo "</dl>" >> foo.html

All we need now is to make the whole process more automatic. Since we have to add that line break in the old export file we can change things up once again to do that automatically. And since we will probably want to save this as a shell script, we can go ahead and make it more readable. I changed a couple of things I didn’t detail here and this is what I ended up with:

First I generalize a bit so I can change things later if I want to

diff $OLDLINKS $NEWLINKS |grep '^> ' |awk '{sub(/<\/A>/,"</a></dt>")};{sub(/<DL><p><DT><A HREF=/,"<dl><dt><a href=")}{sub(/<DT><A HREF=/,"</dd><dt><a href=")};{sub(/<DD>/,"<dd>")}{sub(/LAST_VISIT[^<]*TAGS=/,"tags=")};{sub(/^> /, "")};!/<\/DL>/{print}' > $MYLINKS;echo "</dl>" >> $MYLINKS

then decide on path names (I like to let FireFox save in Downloads automatically and I’m going to delete the new links file anyway, so I set the pathname accordingly.)

export LINKSDIR=$HOME/Documents/Personal/blogging
export OLDLINKS=$LINKSDIR/old-delicious.htm
export NEWLINKS=$HOME/Downloads/delicious-`date "+%Y%m%d"`.htm
export MYLINKS=$LINKSDIR/mylinks.html

then we make our new links file the old one for next week. We should also add that line break while we’re at it (and remove the new links file)

awk '{sub(/<DL><p>/,"<dl>\n")};{print}' < $NEWLINKS > $OLDLINKS
rm $NEWLINKS

and I like to go ahead and open my links file so I can make any quick edits and then post

mate $MYLINKS

I save it and then put it in PATH and make executable

sudo mv preplinks /usr/bin/
sudo chmod 755 /usr/bin/preplinks

You can grab the script here and do the same.

Now every week I go to del.icio.us and export my bookmarks as html and then I run

preplinks

and TextMate launches with my html all ready to be checked and posted.

Works for me.

This is the last in a series of posts. The first two posts are here and here.

Cleaning Up My del.icio.us Links

On Monday, I posted some info about how I am thinking of posting my weekly links.

Today I want to make one correction to the process, talk details about how to clean up the diff file, and then put together a quick script to do that part automatically. Once again, I am going to do this for the first time as I write this. I will summarize the process below.

First, the correction. After my first use of this method I discovered that one more quick edit to the html export will make the parsing of the diff file much easier. Before I move ~/delicious.htm to ~/delicious-old.htm I need to add a line break just after <DL><p>. It may not seem like much but it makes a big difference.

Actually, as it turns out, this is fairly easy to do with awk and grep. Let’s take a look at exactly what I want to do first.

I am only interested in lines that start with > and a space so I start with

grep '^> ' < links.diff

I want to replace the <DL> with <dl> and I don’t need the <p> at all. So now I have

grep '^> ' < links.diff |awk '{sub(/<DL><p>/,"<dl>")}

Now we get rid of the > and the space at the beginning of each line.

grep '^> ' < links.diff |awk '{sub(/<DL><p>/,"<dl>")};{sub(/^> /, "")}

Then we don’t print the last line at all.

grep '^> ' < links.diff |awk '{sub(/<DL><p>/,"<dl>")};{sub(/^> /, "")};!/<\/DL>/{print}'

This gives me everything I need but I still have uppercase tags and attributes, some attributes I don’t really care about, and none of the elements are closed. We can take care of closing the <dl> with a simple echo “</dl>” after it.

echo "</dl>"

So, if we want to save all this to a file we can do this.

grep '^> ' < links.diff |awk '{sub(/<DL><p>/,"<dl>")};{sub(/^> /, "")};!/<\/DL>/{print}' > foo.html;echo "</dl>" >> foo.html

Now all I need to do is clean up those uppercase letters and close all the other elements. I’ll take a look at that on Friday.

This is the second in a series of posts. The first post is here and the next one is here.

Posting del.icio.us Links Weekly to WordPress

I’ve been using del.icio.us to share links since 2005. I’ve always used another method for bookmarking links for myself, but del.icio.us has been my favorite method for the sharing of interesting links. Before del.icio.us I had a separate linkblog so right away I wanted a way to display my shared links in a similar format. I started out by replacing my linkblog with an html rendering of the RSS feed from del.icio.us. I quickly realized that I wanted more than that so I set the blog back up and used a cron job to auto-post my links to the WP database. I’ve written about all of this before.

After a while, I gave up on the linkblog completely and just used a widget to show the links on my blog. Not quite what I wanted but good enough for a while. Recently I decided to set up the daily blog posting feature that del.icio.us provides. This is a very nice feature but doesn’t work well for me because my links come in waves. So, I turned that off earlier this week and set off in search of a way to post the links as a weekly roundup. I’ve seen other sites do this and I like it a lot.

After a few quick searches, I didn’t find anything I thought was worth spending time fooling with. It seems to me it’s just as easy to come up with something on my own. As a hacker I would prefer something as automatic as possible, but I don’t mind having to do something manually. I will probably want to tweak the weekly posting a touch anyway.

The first thing that came to mind was using the RSS feed but I dismissed that because it will only show a maximum of 100 items. That would probably do for my purposes but I’d like to go ahead and set up something I don’t have to worry about – did I get all the links? etc.

So I decided on a different approach. I haven’t done any of this yet. I am going to work on it while I write this.

Here is the plan:

  1. export the links as html
  2. grab out the html I need and paste it into a new post in WP
  3. post it

Simple, except for a few points.

I actually came up with this idea a few days ago and I grabbed an export then. I checked my blog and found that the latest link posted was the trash vortex page at greenpeace.org so I simply removed all links above that and saved this file as ~/delicious.html.

Remove all html above the last posted link

Now it’s time to grab the new links for this week, so I go to del.icio.us and export the html and save it to the desktop. Then,

mv ~/delicious.htm ~/delicious-old.htm
mv ~/Desktop/del*.htm ~/delicious.htm
diff ~/del* > links.diff

The only thing to do now is clean it up and post it. Let’s start by doing it manually. I’ve stripped most of the new links out for demonstration. Take a look.

Diff file

First, I remove the first three lines and the last five lines. I’ve run a few tests now and it looks as though this will always be the case. This should make automation easier. This procedure is obviously going to require a bit of manual intervention so I should be able to notice when a problem crops up.

After removing those lines I am left with a bunch of lines like those below.

> <DL><p><DT><A HREF="http://online.wsj.com/article/SB123731266862258869.html" LAST_VISIT="1238172267" ADD_DATE="1238172267" TAGS="fun,economics,games,culture,scrabble,words">Scrabble and Other Games Have Overvalued Points - WSJ.com</A>
> <DD>Scrabble is a great game and should be left alone.

The only thing necessary to make this “work” is to remove the > at the beginning of each line, but we will make it “right” by changing uppercase tags and attributes to lowercase, closing all elements, and wrapping all of it in

<dl></dl>

Then I copy and paste it into a new post in WP and I’m all set. Requires a bit of input but not hard to do. I will see how much I can automate i on Wednesday.

This is the first in a series of posts. The next post is here

Scraping The iTunes Store

I recently wrote a series of posts detailing the way I chose to scrape iTunes for ratings information for Bailout America, an iPhone game we released recently.

Read more at thedoedoeblog.

LeftLink

LeftLink was a collection of interesting links with re-written headlines. The project slowed to a complete halt due to the time necessary to maintain it manually.

So, it was brought back to life as an aggregation of progressive info using the simple mechanism I put together for iPhoneDeck.

iPhoneDeck

iPhoneDeck uses simple PHP with Magpie to display iPhone related information from RSS feeds.

You can easily set something like this up yourself even if you have very little experience with this kind of thing. Download Magpie and upload the magpie folder to your webspace and get started

I set mine up for someone else to manage so I made it as simple as possible to maintain. I also had to add something to it after it was done – otherwise, I might have done it differently. Here is what I did.

First, I include the magpie library

include('magpie/rss_fetch.inc');

and then I set up two associative arrays – one for the RSS and one for the links to the main pages

$ufeeds = array(
	'Digg' => 'http://digg.com/rss_search?search=iphone&area=promoted&type=both&section=all',
	'delicious' => 'http://feeds.delicious.com/v2/rss/popular/iphone?count=20',
);

$homelinks = array(
		'Digg' => 'http://www.digg.com/',
		'delicious' => 'http://www.delicious.com/',
);

and then I iterate over the data and build out the html.

foreach($ufeeds as $name=>$url) {
	$lname=str_replace(' ','-',strtolower($name));
   	$feed=fetch_rss($url);
	$data=$feed->items;
    echo "<div id=\"$lname\" class=\"news\">\n";
    echo "\t\t<h2><a href=\"$homelinks[$name]\">$name</a></h2>\n";
		echo "<br style=\"clear:both\" />";
		echo "<div id=\"{$lname}-news\" class=\"news-holder\" style=\"height:230px;\">";
	$i=0;
	foreach($data as $item) {
		$i++;
		if($i<21) {
			echo "\t\t<p><a href=\"$item[link]\" class=\"headline\">$item[title]</a></p>\n";
		}
	}
	echo "</div>";

	if($i>10) {
 echo "<div id=\"{$lname}-more\" class=\"more\"><a href=\"javascript:void(0);\" onclick=\"javascript:viewMore('{$lname}');\"  class=\"more\">+ more</a></div>";
 echo "<div id=\"{$lname}-less\" class=\"less\" style=\"display:none;\"><a href=\"javascript:void(0);\" onclick=\"javascript:viewLess('{$lname}');\" class=\"less\">- less</a></div>";
}
	echo "</div>";
} 

Then I wrote the JavaScript functions I needed for the expand functionality.

Then I hired someone to do the CSS. :)

GetMetsTickets.com

I set up my newest website, getmetstickets.com, as a way to help fans who might not otherwise be able to attend.

Learn more about my motivation on my blog.

Turning A List of Links Into One Link

I frequently find myself in need of sending a list of links to a friend. I’m very comfortable with keyboard shortcuts so sometimes, I just do CTRL+space – Te – enter – command+tab – command+l – command+c – command+w – command+tab – command+v – well, you get the idea. Sometimes that method doesn’t really cut it. Sometimes I don’t want to close the tabs as I work which makes the keyboard shortcuts almost complicated enough to resort to using the mouse.

So, I have a couple of other methods using Safari and Automator. Safari is usually my browser of choice anyway so this works for me.

  1. I created a small app using Automator that simply grabs the link from Safari and appends it to a text file. Then I use QuickSilver to call the script and then I close the tab and move on. Of course, if I want to keep the tabs open I modify this a bit but there are still less keystrokes than the method above (even when closing the windows in that method)
  2. I have a modified version of the app I just mentioned that creates clickable links so I can create an html page which I can then either email or post online for a friend. I use three items in Automator.
    1. Get Current Webpage from Safari
    2. Run Shell Script


      I use a Python and pass in the URL as an argument.


      The Python looks like this:

      import sys
      print '<a href="%s">%s</a>' % sys.argv[1]
      
    3. Run Shell Script


      This one is bash

      cat >> ~/Desktop/mylinks.html
      
  3. Today I put together a third method. Last week, I learned about a service called 1link. It wasn’t quite what I wanted and in my search for better I found another service almost exactly like it – linkbun.ch. This still wasn’t quite what I wanted but both services provide an interesting benefit to the receiver of my shared link. You can read more about the services here.

    I decided to throw together a method to give me almost what I wanted. Frankly, this method is not quite as useful to me as what I was already doing but it will come in handy when I want the “one link” benefit for my friends.

    I start with a file with only one line

    links=

    Then I used Automator to build an app that does almost what my other one does, but the Python looks like this

    import sys
    print sys.argv[1]+"%0A"
    

    and the bash

    cat >> ~/Desktop/links.txt

    After I’ve posted all the links to it, I run these two lines

    cd ~/Desktop; curl -d @links.txt http://1link.in/createlink > foo.html; echo “links=” > links.txt

    Then I can open foo.html and click the link. This is better for me than trying to use the web-based form.

AppTheater

AppTheater is a video sharing site centered around iPhone apps. You can watch thousands of iPhone related videos or upload some of your own.

We set this up as a preview of apps. Many of the videos give you a nice demo of the app for your consideration before purchase. You can also review comments and see what other users are saying about the app (or the video).

Check it out at apptheater.com and upload some videos of your own.

If you don’t see an app you’d like to preview, contact me or use the contact form on the site.

Bailout America

I served as program manager for an iPhone app which was recently released at the App Store. The game is called Bailout America (iTunes Link) and it is a real blast.

Check out the game’s homepage here.

Seems to be a hit so far. We’ve had a couple of mentions that I’ve seen online – AppCraver.com and NY Times.

Environment Variables

Here is a quick little Python script I use to inspect environment variables.

Rent Back Direct

Amazing uderstanding of wordpress, this guy really knows what he is doing. excellent work. thanks very much Bill!!

RentBackDirect

Alinks Repairs

Great Programmer! Very easy to work with, good comunication skills, got what I wanted, understood every problem 100%! Highly recommend for your project! ….Thanks!

alinks script repairs and mods

Resurrection

I’ve taken down some of my personal projects in a planned resurrection and renovation.

The first one to come back up will probably be my anti-MLM blog which was a big hit the first time around.

I will keep you posted on my progress.

Code Fragments and Tips

I plan to use this category to post tips and tricks – mostly as a reminder to myself. It would be nice if they are helpful to someone else too.

book

Book post.

WordPress as a CMS

I set up a custom WordPress theme to be used as a complete CMS for this website.

Google Maps – Click to Add

I set up a map for a client who wanted to track firefly sightings in Europe.

I don’t remember all of the specs on this one but I know

  1. Users can click to add a sighting
  2. Users can upload sightings in bulk if they have lat and long data
  3. It uses the lat/long scheme used in the UK
  4. It updates live

It was built with functional and valid xhtml so that it could be placed into another site and styled accordingly. I don’t know where this ended up on the web, but you can see my fully functioning (as 7/12/2007) version at http://mashedpotatoearth.com/fireflies/

ReallyCheckYourself

Boist did a great job on this project. It was completely quickly and accurately. He even went above and beyond to ensure it was done right.


ReallyCheckYourself

Google Maps

I created a Google Maps site which helped users locate medical testing centers. You can see it at http://www.reallycheckyourself.org/.

WordPress Theme

I finished up a custom WordPress theme for this website.

1000 Moms

bsoist offers incredible support & documentation. He has also been very adaptable to a few curve balls thrown at him, and willing to do what it takes to make me happy with the results.

1000 Moms

1000 Moms

I took an existing PHP website set up by a novice and worked in some real programming without disrupting what was already very comfortable for the client. You can see the site at http://www.1000moms1000dollars.com/.

Webpage Replication

This guy rocks!Strongly recommended.Have the exceptional skills required as a programmer to complete my project in time.Very patience and easy to work with.

massive webpage edits

Webpage Replication Service

I wrote a program to edit thousands of static webpages.

Event Tickets Center

I provided a combination of PHP, Perl and JavaScript to solve a cross domain security problem for this website. This was much more difficult than I’ve made it sound but the details are protected by NDA.

Knoxvoice.com

Communicated several times a day on project, responded to emails within hours, completed the project exactly to my original specs. Can’t say enough good things, would definitely work with him again!

Knoxvoice.com Find Us Map

Google Maps API – Knoxvoice.com

I built a “Find Us” map for Knoxville Voice, an independent newspaper in Knoxville, TN.

Find Us

Alinks

I provided several modifications for this script for a couple of different clients.

Betsy Flanagan

excellent work, he’s very conscientious, responsive and upfront. I highly recommend him and will definitely use him again.

–Betsy Flanagan, StartupStudio.com

Startup Studio

I set up and configured a Wordpress powered site for this client. The work included a custom template based on the client’s original design, several plug-in modifications, and two custom plug-ins. The site is inactive now, but you can see it at http://startupstudio.com/.

Data Extraction

This client wanted a list of all YouTube videos on a certain topic. I found some of the work and posted it. Like most scraping work, it may not work anymore. It’s here if you want to check it out.

YouTube Video Mining

Excellent to work with. Great customer service, and willing to do what is necessary to get you what you need. Would work with again. Highly recommended.


YouTube Video Mining

WordPress Theme

I do a lot of WordPress work – themes, plugins, core mods, etc.

This client wanted a standards compliant theme with an adsense style look. He provided an image and I built the xhtml/CSS and the WP theme.

I am not sure where you can find his use of it, but there is a slightly modified version here.

Karl Jackson

Worked hard to get my project functioning. Excellent programmer.

Perl Script Install and Customization

Script Install and Customization

Set up an off-the-shelf Perl script for automated web-based marketing and customized it for this client’s special needs.

Another eBay Listing

Another client needed a custom eBay listing that would get by eBay’s sausage machine.

musicloops.com

Another successful project with bsoist. He managed to quickly solve a problem that two other programmers had much trouble with. Will be using bsoist for all of our large database projects.

musicloops.com

musicloops.com

I solved a query sorting problem for an existing website.

musicloops.com

sound-effect.com

bsoist is amazing. He took our messed up existing code and turned into a work of art. He is fair, fast, honest, great communication, everything you could want in a coder/designer. We will be using bsoist for all of our future projects.

sound-effect.com

sound-effect.com

I built a database driven website which provides free sound effects and other clips.

sound-effect.com

osCommerce Automation

Another client who wanted to automate the process of delivering finished websites to his customers.

This client sold complete osCommerce sites pre-loaded with inventory and I wrote an application that would accept FTP credentials as input and fully automate the process where the host supported SQL access http. If not, a database could be created first and then the process would work.

M@D 2 Host – WHM Integration

I setup WHM integration for this web hosting company.

M@D 2 Host

eBay Listing

Worked very hard on the projct.Reliable and easy to communicate with. Will definitely work with again.

eBay Listing

eBay Listing

I created xhtml/CSS that would create the custom look the client wanted and survive eBay’s sausage machine.

Listings no longer active.

PHP PayPal Functions

This client wanted very a very specific layer of abstraction for use with several of his websites.
I am not sure how useful it would be to others, but if I can find it I will post it here.

Layout Jerk

Great work, and will be using for more work in the near future.

Layout Jerk

Layout Jerk

I created hundreds of layouts for MySpace pages for this site.

Layout Jerk

WordPress Modificaitons – selfinvestors.com

Bill dealt with a project that was was far more difficult and time consuming than imagined with great professionalism and patience, continuing to work to find solutions. Clearly, Bill is highly experienced programmer with wordpress expertise. I would use him again.

selfinvestors.com

WordPress Modifications – selfinvestors.com

I wrote a few custom plugins and theme modifications for this website.

selfinvestors.com

Dynamic Memory Solutions

I ported this software for several different *nix shells.

DMS Website

Website Replication

A guru in every sense of the word! This guy should be given a goverment national asset award for services to the United States! Hire him now!

Website Replication

Website Replication

My work seems to come in themes and this was the first in a long line of automation for sales of websites.

This client sold new websites to clients – what has become known as splogs (spam blogs) – and he wanted an easy way to replicate them on demand.

That’s where I came in. :)

Scraping Real Estate Data

I created an application that would mine all the data on this website and return a complete list of all property meeting a certain criteria. I provided this to the client as a Windows executable which he could run whenever he wanted to search. He was looking for very specific properties, but the application allowed for different searches on demand.

I’ve actually sold derivatives of this application to several clients.

Moodle Customization, PayPal Integration – Paid Subscriptions

I created an online tutoring website based on Moodle. I set up PayPal subscriptions for class dues.

http://tutoringsessions.com/

SiteReportCard.com

I wrote the application that powers this website. I did not design or build the website, but I provided all the code that grabs the data, parses it, tabulates the results and presents it.

I enjoy this kind of work very much and it is one of my specialties.

Rebuilding Pages

You sir are a genius. This is going to save SOOO much time and it looks like we’re down to the final bit.

Did a really good job getting my script put together with very little to work with! Easy to get ahold of via IM and replies to emails quickly. No complaints at all.

parsing and rebuilding pages

Parsing and Rebuilding Pages

I wrote a program that would parse and rebuild 1000s of existing webpages. This project involved programmatically “moving” pages to different sections of the website, changing names and several other details on every page.

I don’t have a link.

News Website

This is A+++ programmer. Nice to deal with him. Good knowledge and worth capable. More projects for him are waiting. Thanks.

I created a news website for this client. The site is no longer in operation.

Data Extraction – Sports Scores

I extracted years of historical data – NBA box scores – for a private client. I also wrote a program that would extract current data as needed.

I found an unfinished version of this script over here.

Matthew Oswald

Great programmer, writes very clean code. Very willing to troubleshoot. I will, without a doubt , work with him again!

Google Search API

Google Search API

Matthew hired me to use the Google search API to find the number of results ( the real number, not the number Google reports ) for each keyword in his database and then insert that number in another column in his database.

There is no web based user interface to link to.

ThirdSphere :: Web hosting for small business success! | bsoist’s work

Fantastic to work with this gentleman. Does his job well and goes above and beyond. I will definately use him again. Infact, I have already offered him another project.

ThirdSphere :: Web hosting for small business success! | bsoist’s work

ThirdSphere :: Web hosting for small business success!

I was hired to transfer several websites from one server to another.

ThirdSphere :: Web hosting for small business success!

Sport.co.uk – Sport Resources and Information.

I was hired to build a complete sports website in the UK. Site is no longer in operation but you can see it in the wayback machine.

Sport.co.uk – Sport Resources and Information.

PhysOrg.com: latest science and technology news

I was hired to write an application that would pull RSS feeds and parse into PHP arrays.

PhysOrg.com: latest science and technology news

Greg – Music Utopia

bsoist is a very good programmer, who worked on my project to 110% of my satisifaction. The programming he did was clean, and works great! He preformed the project in a very reasonable amount of time. His communication throughout the project was outstanding. I recommend him to everyone and I will be using him again my self. Thanks for the great job bsoist!!

Music Utopia

Music Utopia

Greg hired me to add the storefront to his indie music website.

Music Utopia Home Page

ultrabrightlights.com

Was very helpful and very persistant on getting our problem resolved.

ultrabrightlights.com

PayPal Integration – ultrabrightlights.com

I setup PayPal IPN and streamlined the shopping cart process.

ultrabrightlights.com

Surreal Art, Fantasy Art & The Contemporary Surrealism of Domen Lombergar

Domen hired me to add a couple of new features to his website and fix up some SQL problems.

Surreal Art, Fantasy Art & The Contemporary Surrealism of Domen Lombergar

Domen Lombergar

Very qualified programmer. Provided lots of feedback during the creation process and created a marvelous job in the end. Highly recommended.

Surreal Art, Fantasy Art & The Contemporary Surrealism of Domen Lombergar

Amazon API

I’ve been experimenting with Amazon.com’s APIs for several projects.

You can see my work here.

Experience