In cyberspace, no-one can hear you scream...

Thankfully. After every form of testing you can think of, for no real reason, our database decided to crash. Perfectly repairable, but a pain nonetheless. Sorry if you were widgetless for a bit.

Version 2 launched

Please God it works!

You'll see the new version on the left hand side.

Some major changes:

1. No more auto-scrolling list. We never liked it, but it required a major soak in the shower to sort it out.

2. A tag cloud by default. People love tag clouds. They are visual and accessible. We put a lot of work into getting the colors to be complementary to the base text color.

3. Better tagging algorithm. Again, a lot of work went into this, with the introduction of "SuperTags", which you will see are in parentheses. These are non-literal, ie they are an attempt to work out what the page is about, not just taking interesting phrases

4. Better scoring and refinement. Try limiting the number of tags on a page, and you'll hopefully still get what is useful

5. Pick a URL or Twitter feed. Now you can choose any URL or Twitter feed, and have those tags display on your page

6. List view on a tab. This is how we sorted the auto-scroller, by hiding it behind the cloud (and not letting it scroll automatically)

7. Related categories. If you click on a tag and it has any Wikipedia categories associated with it, there is an option to make them appear.

8. IE8 Accelerator. Not strictly part of the widget, but check it out at http://linkky.com. If you use IE8, you can get a little tag cloud of the page you're viewing, just using a right-click.

To Do:

1. Code cleaning. It's all a bit slower than it should be, mainly due to some redundant code

2. Taggregation. Don't you wish you could mash up a few pages/feeds, and see what they all have in common - Coming soon!

3. Image only tag cloud. For those of you who use Wordpress.com and can't add widgets. We hope to have an image URL you can use to have a nice picture of what a tag cloud would look like on your page...

Linkky goes all Google Gadgety

With a bit of tweaking, we found out how you get your widget into IGoogle and other such places:

1. Get the widget.

2. Copy the "Embed" code

3. Paste it into Notepad or a similar text editor. It will look a bit like this in the middle:

WIDGETBOX.renderWidget('b14saa0c-ab64-4c9b-be28-a604bcf8e42e')

4. Copy the long ID (eg b14saa0c-ab64-4c9b-be28-a604bcf8e42e)

5. Insert it at the end of this this URL in place of b14saa0c-ab64-4c9b-be28-a604bcf8e42e

http://widgetserver.com/syndication/GetWidget/googlegadget/b14saa0c-ab64-4c9b-be28-a604bcf8e42e

6. Paste the new URL anywhere you can insert a Google gadget (eg iGoogle or GMail using the new Labs feature)

Twitter U like

Yes, you can now enter a Twitter ID into the widget, and get tags generated from that Twitter feed. Just edit the widget, and look at the bottom of the settings page.

What's in beta

You may have noticed a separate widget on the bottom left of the screen, which changes periodically. And sometimes doesn't show up at all!

We're experimenting with two new features at the moment.

First is the continuous scrolling widget. Before, you had to wait for the whole list to go past, before you could see the rest of the list, which is a bit of a pain, frankly. The continuous scrolling part we seem to have cracked, in part. The problem is getting the expanding boxes right.

The second element is to cut out the AJAX call. Because of latency issues, the existing widget scans the page, and throws up an initial list of topics. It then goes off to wikipedia, and looks to see if there are any matches, and then throws the results back. Technically interesting, but ultimately, not ideal. Fortunately, wikipedia has vast dump files that you can play around with. So over the weekend, we stripped out all the information we (think we) need, so that it will be a one step process of putting the widget out. It just sounds so easy. Right

Finally, there is the bizarre label thing in the posts, which currently say "Beatles1". This is an experiment is matching labels to posts using Feedburner. In time, we'll be able to do it. The biggest issues is that they are produced using Javascript, and so aren't picked up by Technorati. Yet.

Quick label test

Just a label test

That new widget in full

The changes are subtle, but useful.

The big improvement is that you can now have results returned that only match Wikipedia categories. That way, you do know that you are getting information into your widget that will benefit your users. The downside is that you might miss stuff.

You can configure the widget using the little pencil tool at the bottom the of the widget box, and move some of the parameters around to get the best results for your site (you can also do strange things with the Widgetbox smart tags to make each widget instance different for each page, but we've noticed that off things then happen on archive pages - Back to the drawing board on that one)

Technorati Integration

Well, we're looking at seeing how we can get the categories we create to link in to Technorati.

Just tried to claim this blog using the Technorati claim service, and got one in Japan! So here's another try...

Technorati Profile

Almost beaten by a beta...

We're always trying out new ideas here to improve not only the quality of our results, but how we present them, and how usable we make them.

The latest experiment is around reshuffling the results if they coincide with a Wikipedia category, and pushing them to the top, and possibly even ditching the rest (up to the user)

It almost killed us working out why it wasn't working. Two days later, we found we were missing an equals sign. Honestly, the easy stuff seems hard some days. Why can't the hard stuff compensate by being easy?

Has your widget just gone screwy

Or why sometimes using someone else's "robust" technology really sucks, and you wish you'd done it your way in the first place.

We use Widgetbox for our widget. It is a selling point, as it is stable, fast to load, doesn't interfere with your blog etc etc.

Except that they've just updated the management interface. And making a small change has screwed all our widgets up. If it's happened to you, just go into the configuration page (click on the pencil under your widget), and look down the list. Most likely the language code has appeared in your maximum results fields, but there may be more.

As Jon just posted on the Widgetbox support forum:

"This is more of an observation than anything, although could do with being fixed.

I just added a new parameter to my widget (quick plug for Linkky), using the new edit functionality.

In doing so, it has shifted some of the values for my already published widgets (so my language code got moved to my maximum results field, for example)

This means that anyone who has my widget installed will now find they have to go back and manually readjust some of their settings. At best annoying. At worst, it will lose me users who find their widget no longer works properly

Fortunately, I currently have a manageable number of subscriptions. But if you have a few thousand, they will all get screwed up if you add a parameter...

Jon"

The last 10%

Does take 90% of the time.

We're doing a lot of fiddling to iron out a few inconsistencies, and to ensure that everything that should get linked, does, and that everything that shouldn't, shouldn't.

We'd hoped to add the ability to have more of less tags to the mobile service, but that's taking longer than we hoped. That's what happens when you use some third party code (no matter how good it is, you have to work out how it works, and make it do things the way you would)

Hopefully the mobile service can be properly soft-launched later...

Widget V2

Time to send the new baby live!

Lots and lots of new features in this.

The main one is that once the initial topic set comes up, the system goes off and sees if there are any related categories elsewhere that they might fit in to (based around the Wikipedia categories), and stick them in a sub- folder

You can click on any link, and it will now return a list of possible searches, currently site search, Google search, Wikipedia and Amazon (Google and Amazon have some IP sensitivity as to which one country you see)

The widget can also now be any size, and the scroll can be speeded up or slowed down

Caching will be on, so the results will stay the same for eight hours at a time. We'll be looking at pinging over time to try to improve that

All three bits seem to work

If you go to http://www.linkky.com, it does seem that the mobile proxy, the widget and the Xtractor function all work. The caching was a nightmare, but it was eventually solved using a home-brew solution, having spent hours fiddling with mod_rewrite and mod_cache, which is alleged to now work, but didn't.... It would be more elegant than the way we're doing it, but hell....

URGH. Late night....

Hopefully ironed some of the bugs out. Hours of fun with arcane code. It was our intention that a new widget would be generated every 8 hours, to keep the server load down, but the changing timestamp sent by Widgetbox meant that the server keeps thinking its a new request, and going out and rescanning the blog pages over and over again, rather than waiting

This post in itself will hopefully be a test of whether its fixed or not!

Site's sort of up

The website at http://linkky.com proceeds. The Xtractor function is pretty much there, although it gives out random widgets and the HTML it outputs is ghastly (although that will be good for anyone trying to extract meaning from what come our without too much parsing!)

Hopefully the DNS will be happy by tomorrow.

Bloody caching's going to be an issue. The Widgetbox parameter string changes every time...

Server's up

The server works, and seems to be serving up widgets at an appropriate rate. The widget is ready to go out into the gallery, although I'd prefer it if we had the web-site up and running. I have one enduring query for the developers which is around how we marry up some of the searching.

In deciding what goes into the mix for creating search terms, we're till picking up
some page furniture, and sometimes that causes an issue with a false result. Doesn't stop you doing a Google or Wikipedia search, though.

We haven't got round to putting in the "suggest a deletion" button yet, but you can now vary the number of search terms, so that may help.

New server

Has been bought, but that's about it so far.

Biggest issue at the moment is trying to find a suitable strap line to explain what it is we do. We take a web page, shake out the interesting phrases, and let you search through that site, or Google or Wikipedia, without having to bring up the search bar to do it.

How to do you get that down to 3 words?!

Hurray! We got the domain name

Difficult things to choose - You want something descriptive, but can't always get it. Linkky was about to expire, and it wasn't being used, so that was the one for us!

Linkky

This is a quick intro for what Linkky is going to be all about.

It has a few features, but all of them are concerned with extracting interesting stuff from content, and making it useful.

For example, you should be able to see a Widgetbox hosted widget alongside this blog with a scrolling list of phrases and words. Click on one, and you'll do a search of this whole blog for that phrase. Click on one of the icons, and search more widely, or suggest that it's rubbish, we should delete it from the database.

There will also be the "shrink and link" service that allows you to surf the web from a mobile phone and have all of the extraneous rubbish stripped from the pages to make it quicker to surf (and use less bandwidth), but also have important words and phrases hyperlinked, so that with one click you can go search the web for that subject. So much easier than trying to type in the URL of a search engine from your phone, and then enter the search term.

More things will be added over time.

We really appreciate any comments you have, in particular over the quality of the results we serve up. There is a lot of back-end processing that takes place to try to find what's relevant, and with your feedback, we can make that so much better.