Hey everyone, I wanted to run an idea past you guys to see if you would find it useful. I have developed a service that runs every fifteen minutes on my computer and checks whether every non-retired LEGO set is available. If not available, it changes my database to reflect that and sends a text message and an email to me to alert me of the retirement. It has been working well and accurately for the last 4 months (when I first developed it). I was curious as to whether anyone else on this forum uses a similar technique and whether you guys think this is a good idea for a program. Also, the obvious bottom line question is whether anyone thinks this kind of idea is worth any money. I was considering having some sort of subscription that I could have people buy to be added to the distribution list. My service runs every 15 minutes, so when it finds retired sets, it sends an email/text to everyone on the distribution list. By being on the distribution list, you are guaranteed to know within 15 minutes of when a set goes EOL. You then have the advantage of being one of the first people to hop on Amazon/Ebay and buy up the remaining stock wherever available. So the real reason behind the program is to get an advantage over other collectors by receiving alerts before they do. Let me know what you guys think moving forward. I also was curious as to whether there was anything like this out there already or whether mine is the first to be brought forward. Thanks, mrkp

In theory, this sounds like an awesome idea. But the problem I am seeing right away is that there is no concrete way to know 100% for sure when something officially goes EOL. The only way that I know of is to look at LEGO LEGO Shop at Home and see if it says retired product. Other than that, EOL dates are pretty much locked up in Fort Knox.

The only way that I know of is to look at LEGO LEGO Shop at Home and see if it says retired product.

I agree, this is the only concrete evidence we get of something going EOL, but most sets that go EOL don't even get this notice, so we kind of have to guess. And then, there are the seasonal sets like the Holiday sets and maybe Halloween sets, they can disappear for months and then reappear.

what website and what indicator are you using to see if its available. Is it just the shop.lego.com and scraping the word available or something like that?

...

I wanted to run an idea past you guys to see if you would find it useful. I have developed a service that runs every fifteen minutes on my computer and checks whether every non-retired LEGO set is available. If not available, it changes my database to reflect that and sends a text message and an email to me to alert me of the retirement. It has been working well and accurately for the last 4 months (when I first developed it).

...

Are you basically "spidering" LEGO website, or do you spider more than just LEGO.com? I've written scripts in the past that collects data from the entire database from a site and expanded upon that with our own data, so I am curious about your approach.

Also what criteria are you using to determine if a set is "retired"? I suspect Imperial Shuttle (10212) is already retired, but it says nothing of such nature on LEGO.com. Just "out of stock". If it's truly retired, at this point, its already too late.

I am essentially scraping the LEGO Shop at Home page but in a slightly more informed way than you have suggested. I will not give specifics if this does ever get transformed into a better idea, but my retire dates in my database for the last 4 months match pretty closely to what is in brickset has for the same sets and their availability. Rest assured, it is more complex than loading a page and searching for 'out of stock' or 'available' within the content of the page. As far as 10212 is concerned, by my technique, it is not retired yet. 'Out of stock' doesn't equal 'EOL' by my method. Although it may go EOL soon, it is not yet for me. - mrkp

I am essentially scraping the LEGO Shop at Home page but in a slightly more informed way than you have suggested. I will not give specifics if this does ever get transformed into a better idea, but my retire dates in my database for the last 4 months match pretty closely to what is in brickset has for the same sets and their availability.

Rest assured, it is more complex than loading a page and searching for 'out of stock' or 'available' within the content of the page.

As far as 10212 is concerned, by my technique, it is not retired yet. 'Out of stock' doesn't equal 'EOL' by my method. Although it may go EOL soon, it is not yet for me.

- mrkp

Then this should be an interesting test. Imperial Shuttle now has "retiring soon" tag on LEGO.com. Time will tell and if IS is indeed retired now, then your technique has not caught on to this fact. I've not seen any IS at retail anywhere using Zoolert for about a week now.

If IS stock is replenished anywhere at retail with any degree of regularity (ie: not just about a dozen that people happened to return to Amazon that will of course get sold), and your technique still manage to nail the later retirement date, then color me impressed.

Not sure why you can't share the details. I hope you don't plan to charge people for this service? Or are you simply in fear that other people will catch on to how your technique works and deny us the chance of scoring sets before they retire?

Not sure why anyone is doubting that the IS is EOL.....because they didn't scoop one up when they were available maybe???

Not sure why you can't share the details.

he can share with me.......I am no good with algorithms :-)

Ok, well this thread turned out to be worthless. All I can say is, if you find something that works for you, then it's a feather in your cap. Good luck.

my retire dates in my database for the last 4 months match pretty closely to what is in brickset has for the same sets and their availability.

The problem here is that Brickset "retires" a set every time it goes out of stock at LEGO Shop at Home. Then the availability dates update at Brickset when it is back in stock. I love Brickset as a reference tool, but it is nowhere near an authority on EOL status and dates.

Exactly, i haven't bothered looking at that page for a long time. Even Huw knows its not accurate, but its the closest thing we have. Even if I were to create a list here, do we know for a fact that Imperial Shuttle is indeed retired at this point. I can't say for sure. LEGO is the only one that knows this info. I can create a feature like that here but I honestly wouldn't know what good it would do. Also if LEGO notices you are pounding their site every 15 minutes, there is a chance in time they could block your IP and then that is the end of your feature unless you are going to constantly change IPs. I don't want LEGO being mad at me, its not worth the risk.

The one thing about this as well is that if you are into LEGO investing, you should be well aware of what sets are coming an going. The entire time you should be looking for deals and sets in your inventory. Seeing if a set is now EOL on LEGO should not become a surprise. For example Imperial Shuttle has been on a "watch list" for quite a while, we all knew it was coming close to the time. There were some great deals on it not too long ago and that's when people should have added some to their inventory. Having an alert that tells you it went into retired mode in my opinion is too late.

The one thing about this as well is that if you are into LEGO investing, you should be well aware of what sets are coming an going. The entire time you should be looking for deals and sets in your inventory. Seeing if a set is now EOL on LEGO should not become a surprise. For example Imperial Shuttle has been on a "watch list" for quite a while, we all knew it was coming close to the time. There were some great deals on it not too long ago and that's when people should have added some to their inventory. Having an alert that tells you it went into retired mode in my opinion is too late.

I agree...I think people are putting to much emphasis on retired or not retired. If you feel it's a good set for investment, you better buy them earlier than later. As Jeff said, there have been plenty of deals on these sets. I saw the Imperial Shuttle for less than $200 many times. Just like the Grand Emporium, Fire Brigade, Diagon Alley, Death Star, SSD...whatever. There have been some great deals. People need to stop waiting to see 40% off for these quality sets because you will not get it, contrary to some posts I read. There is a lot of exaggeration on the forums about finding great deals in stores, so people wait, hoping to cash in. By the time some of you pull the trigger, it will be too late.

The EOL data is overrated information to the smart investor to be honest. The smart investor will have already bought the sets they wanted at great prices. There are many sets that get discounted early on, then never again. You see a good deal...go for it. Otherwise, you might be paying reseller prices after it is retired for the fifth time. ;-)