Posting del.icio.us Links Weekly to WordPress
I’ve been using del.icio.us to share links since 2005. I’ve always used another method for bookmarking links for myself, but del.icio.us has been my favorite method for the sharing of interesting links. Before del.icio.us I had a separate linkblog so right away I wanted a way to display my shared links in a similar format. I started out by replacing my linkblog with an html rendering of the RSS feed from del.icio.us. I quickly realized that I wanted more than that so I set the blog back up and used a cron job to auto-post my links to the WP database. I’ve written about all of this before.
After a while, I gave up on the linkblog completely and just used a widget to show the links on my blog. Not quite what I wanted but good enough for a while. Recently I decided to set up the daily blog posting feature that del.icio.us provides. This is a very nice feature but doesn’t work well for me because my links come in waves. So, I turned that off earlier this week and set off in search of a way to post the links as a weekly roundup. I’ve seen other sites do this and I like it a lot.
After a few quick searches, I didn’t find anything I thought was worth spending time fooling with. It seems to me it’s just as easy to come up with something on my own. As a hacker I would prefer something as automatic as possible, but I don’t mind having to do something manually. I will probably want to tweak the weekly posting a touch anyway.
The first thing that came to mind was using the RSS feed but I dismissed that because it will only show a maximum of 100 items. That would probably do for my purposes but I’d like to go ahead and set up something I don’t have to worry about – did I get all the links? etc.
So I decided on a different approach. I haven’t done any of this yet. I am going to work on it while I write this.
Here is the plan:
- export the links as html
- grab out the html I need and paste it into a new post in WP
- post it
Simple, except for a few points.
- How do I know what html I need?
I decided to use a simple diff between the exports from this and last week. Seems simple enough.
- I’d like valid xhtml and it doesn’t look like it comes out that way
I should be able to take care of this with some regex.
I actually came up with this idea a few days ago and I grabbed an export then. I checked my blog and found that the latest link posted was the trash vortex page at greenpeace.org so I simply removed all links above that and saved this file as ~/delicious.html.
Now it’s time to grab the new links for this week, so I go to del.icio.us and export the html and save it to the desktop. Then,
mv ~/delicious.htm ~/delicious-old.htm
mv ~/Desktop/del*.htm ~/delicious.htm
diff ~/del* > links.diff
The only thing to do now is clean it up and post it. Let’s start by doing it manually. I’ve stripped most of the new links out for demonstration. Take a look.
First, I remove the first three lines and the last five lines. I’ve run a few tests now and it looks as though this will always be the case. This should make automation easier. This procedure is obviously going to require a bit of manual intervention so I should be able to notice when a problem crops up.
After removing those lines I am left with a bunch of lines like those below.
> <DL><p><DT><A HREF="http://online.wsj.com/article/SB123731266862258869.html" LAST_VISIT="1238172267" ADD_DATE="1238172267" TAGS="fun,economics,games,culture,scrabble,words">Scrabble and Other Games Have Overvalued Points - WSJ.com</A>
> <DD>Scrabble is a great game and should be left alone.
The only thing necessary to make this “work” is to remove the > at the beginning of each line, but we will make it “right” by changing uppercase tags and attributes to lowercase, closing all elements, and wrapping all of it in
<dl></dl>
Then I copy and paste it into a new post in WP and I’m all set. Requires a bit of input but not hard to do. I will see how much I can automate i on Wednesday.
This is the first in a series of posts. The next post is here
Scraping The iTunes Store
I recently wrote a series of posts detailing the way I chose to scrape iTunes for ratings information for Bailout America, an iPhone game we released recently.
Read more at thedoedoeblog.
LeftLink
LeftLink was a collection of interesting links with re-written headlines. The project slowed to a complete halt due to the time necessary to maintain it manually.
So, it was brought back to life as an aggregation of progressive info using the simple mechanism I put together for iPhoneDeck.
iPhoneDeck
iPhoneDeck uses simple PHP with Magpie to display iPhone related information from RSS feeds.
You can easily set something like this up yourself even if you have very little experience with this kind of thing. Download Magpie and upload the magpie folder to your webspace and get started
I set mine up for someone else to manage so I made it as simple as possible to maintain. I also had to add something to it after it was done – otherwise, I might have done it differently. Here is what I did.
First, I include the magpie library
include('magpie/rss_fetch.inc');
and then I set up two associative arrays – one for the RSS and one for the links to the main pages
$ufeeds = array(
'Digg' => 'http://digg.com/rss_search?search=iphone&area=promoted&type=both§ion=all',
'delicious' => 'http://feeds.delicious.com/v2/rss/popular/iphone?count=20',
);
$homelinks = array(
'Digg' => 'http://www.digg.com/',
'delicious' => 'http://www.delicious.com/',
);
and then I iterate over the data and build out the html.
foreach($ufeeds as $name=>$url) {
$lname=str_replace(' ','-',strtolower($name));
$feed=fetch_rss($url);
$data=$feed->items;
echo "<div id=\"$lname\" class=\"news\">\n";
echo "\t\t<h2><a href=\"$homelinks[$name]\">$name</a></h2>\n";
echo "<br style=\"clear:both\" />";
echo "<div id=\"{$lname}-news\" class=\"news-holder\" style=\"height:230px;\">";
$i=0;
foreach($data as $item) {
$i++;
if($i<21) {
echo "\t\t<p><a href=\"$item[link]\" class=\"headline\">$item[title]</a></p>\n";
}
}
echo "</div>";
if($i>10) {
echo "<div id=\"{$lname}-more\" class=\"more\"><a href=\"javascript:void(0);\" onclick=\"javascript:viewMore('{$lname}');\" class=\"more\">+ more</a></div>";
echo "<div id=\"{$lname}-less\" class=\"less\" style=\"display:none;\"><a href=\"javascript:void(0);\" onclick=\"javascript:viewLess('{$lname}');\" class=\"less\">- less</a></div>";
}
echo "</div>";
}
Then I wrote the JavaScript functions I needed for the expand functionality.
Then I hired someone to do the CSS. 
GetMetsTickets.com
I set up my newest website, getmetstickets.com, as a way to help fans who might not otherwise be able to attend.
Learn more about my motivation on my blog.
Turning A List of Links Into One Link
I frequently find myself in need of sending a list of links to a friend. I’m very comfortable with keyboard shortcuts so sometimes, I just do CTRL+space – Te – enter – command+tab – command+l – command+c – command+w – command+tab – command+v – well, you get the idea. Sometimes that method doesn’t really cut it. Sometimes I don’t want to close the tabs as I work which makes the keyboard shortcuts almost complicated enough to resort to using the mouse.
So, I have a couple of other methods using Safari and Automator. Safari is usually my browser of choice anyway so this works for me.
- I created a small app using Automator that simply grabs the link from Safari and appends it to a text file. Then I use QuickSilver to call the script and then I close the tab and move on. Of course, if I want to keep the tabs open I modify this a bit but there are still less keystrokes than the method above (even when closing the windows in that method)
- I have a modified version of the app I just mentioned that creates clickable links so I can create an html page which I can then either email or post online for a friend. I use three items in Automator.
- Get Current Webpage from Safari
- Run Shell Script
I use a Python and pass in the URL as an argument.
The Python looks like this:import sys print '<a href="%s">%s</a>' % sys.argv[1]
- Run Shell Script
This one is bash
cat >> ~/Desktop/mylinks.html
-
Today I put together a third method. Last week, I learned about a service called 1link. It wasn’t quite what I wanted and in my search for better I found another service almost exactly like it – linkbun.ch. This still wasn’t quite what I wanted but both services provide an interesting benefit to the receiver of my shared link. You can read more about the services here.
I decided to throw together a method to give me almost what I wanted. Frankly, this method is not quite as useful to me as what I was already doing but it will come in handy when I want the “one link” benefit for my friends.
I start with a file with only one line
links=
Then I used Automator to build an app that does almost what my other one does, but the Python looks like this
import sys print sys.argv[1]+"%0A"
and the bash
cat >> ~/Desktop/links.txt
After I’ve posted all the links to it, I run these two lines
cd ~/Desktop; curl -d @links.txt http://1link.in/createlink > foo.html; echo “links=” > links.txt
Then I can open foo.html and click the link. This is better for me than trying to use the web-based form.
AppTheater
AppTheater is a video sharing site centered around iPhone apps. You can watch thousands of iPhone related videos or upload some of your own.
We set this up as a preview of apps. Many of the videos give you a nice demo of the app for your consideration before purchase. You can also review comments and see what other users are saying about the app (or the video).
Check it out at apptheater.com and upload some videos of your own.
If you don’t see an app you’d like to preview, contact me or use the contact form on the site.
Bailout America
I served as program manager for an iPhone app which was recently released at the App Store. The game is called Bailout America (iTunes Link) and it is a real blast.
Check out the game’s homepage here.
Seems to be a hit so far. We’ve had a couple of mentions that I’ve seen online – AppCraver.com and NY Times.
Experience
automation awk bash Books Code csh eBay Google API Google Maps grep JavaScript ksh magpie Moodle osCommerce parsing PayPal API Perl PHP Python RSS SQL WordPress xhtml/CSS XML-
Good Books
