Ooof… just ooof.
So, i redid the Amazon filament script a few days back. That one is a really “heavy” script. It starts off downloading data on about 20,000 items (just about everything filament with free shipping), it then does a few loops back to get additional data to help filter everything out. Overall it uses about 3-4,000 api calls, and takes about 45 minute to run from beginning to end.
So, after completing that monster, i got ballsy…. decided to see if i can do something similar for AliEx. And like i said in the very beginning – ooof.
AliExpress has gotten much better over the years, but it still has that one thing that drives everyone insane – the switcharoo. You see an item in browse you like at a price you like, you click on the thumbnail, and when you get to the listing it’s a completely different price, sometimes on a completely different item… and to add insult to injury, even though you had free shipping selected in browse, the item has an insane shipping fee. That’s pretty much the AliEx browsing experience (Ebay is not much better).
A while back i tried their api. It wasn’t much better than browse. In fact, it was literally the same thing. Same listings, same sellers, same switcharoos, and no way to tell if something has shipping until you actually click on the listing. Wasn’t worth the effort, i gave up pretty much instantly.
Turns out (thanks to ChatGPT) that they have another “secret” advanced version of this api. This one requires a special application/approval, but i got it. It’s very much experimental, a little bit buggy, and very very VERY slow. There is only one server (which i can only i assume is a potato) located all the way in Singapore. Every request needs to be spaced out at least 30 seconds apart, or it will get throttled and eventually banned for a bit. This doesn’t count retries, etc. My test runs of about 400 items tend to take 5 hours or so, most if it spend waiting to make the next call… there’s maybe a grand total of 30 seconds worth of actual processing.
So, incredibly slow…. but working 🙂
And by working i mean data is coming in. The quality of the data is…. let’s just once again refer to my previous – ooof. Actually, i’m being harsh – the data itself is fine, it’s what’s in the data that’s the problem, or rather lack of proper data. There is no weight, color, etc. So to figure out what i’m looking at, i had to setup all kinds of regex conditions, etc. Basically i’m guessing the weight from titles, and variation names.
It’s far from a perfect system, but it’s all i got. It’s still misfires quite often, mostly because sellers just do a terrible job of providing info. Right now the number one scored item and best price/kg is a 2 pack from Elegoo that is incorrectly being identified as a 10 pack. There is not much i can do to fix that (though i will try again tonight), simply because there is nothing on that listing to indicate that it’s 2kg. The way that listing is setup is just deceptive. I may end up blacklisting it alltogether.
So, there will be an occasional false positive that slips through the filters here and there, but for the most part – it works, and does what it should. It picks up a list of what AliEx has on sale for filament from the trusted sellers (they’re not the best, but they’re the best AliEx got), filters out anything that’s not free shipping to/from US, and gets prices for all variations of a product/listing.
It then picks up the top 200 sorted by score (i’ll get into that some other time), and generates the report post. This will run daily, though it will not always update daily. It will check for new deals daily, but will refresh/update existing deals every 2-3 days (AliEx prices don’t jump around that often) to help limit the run time at least a little bit.
So yeah… we now have AliEx deals. Ebay is next 🙂
