today we're continuing our quest to bring back the world's first search engine called Archie this once prolific service critical to users of the early internet has seemingly vanished Without a Trace so join us as we go on an epic journey to find and resurrect the search engine that started it all Archie is gone this once prolific service of the internet has vanished there are no Archie servers still online and All That Remains are ghostly remnants of a bygone era scattered across the web on forgotten web pages and FTP itself is hanging on by a thread major web browsers no longer support the protocol and once popular and thriving FTP sites are now shutting down which is a shame we think such an important piece of of Internet history should be preserved and so we've decided to try to run an Archie server ourselves FTP was such a huge part of the 90s internet experience and bringing Archie back seems like the perfect way to recapture that magic but with Archie long since abandoned we're going to have to figure out where to find the software we first turn to Alan Mage with our idea there is a copy supposedly at the computer Museum in um where is it Palo Alto or Mountain View something Mountain View Mountain View yeah you might be able to get try and get hold of them and see if they were they were given the tape of the last known tape of the source of orgy um but I never heard back from them on whether or not they were actually able to you know attract get it off yeah y yeah the Computer History Museum's collection contains over 150,000 items and luckily it is searchable through their website aside from an information super Highway training video from 1994 there is nothing relevant here to Archie so it could be that the tape copy that Mage provided was not recoverable finding information on the web about Archie's early days is difficult since the Wayback machines earliest crawls are from 1996 several years after Archie's Peak we did find a record of bunny up.com from 1999 just before the company closed its doors from this page we found that Archie's last version was 3.5 and it was selling for $6,000 a one-time fee for an unlimited license but there was a discount for educational institutions we also found a listing of Archie servers from 1997 and up through 1999 but none of these servers are still online their telnet interfaces have been shut down now for most likely well over a decade so where does this leave us in our search well there aren't going to be any easy answers if we're going to find Archie we're going to need to scour the corners of the web in the last remaining vestages of the early internet as you might imagine a popular service like Archie spawns several other projects that aim to expand its access and make it easier to use so we thought starting there might get us some leads of where to find Archie the first we found was an Archie client for the Macintosh called Anarchy it did away with the need to connect to Archie servers through a command line telnet client and instead provided a graphical user interface along with a built-in FTP client for easy file downloads we found a contact for the program's Creator Peter Lewis and amazingly he responded with an old Archie server listing query code as well as the transcript of an actual Archie session but Lewis did not have a copy of Archie itself we then found the creator of archiplex mar coaster archiplex was a worldwide web gateway to Archie servers that allowed users to query different Archie servers from our web browser and the CGI form interface is still available to see but with no Archie servers left it doesn't actually work but again we hit a dead end as coaster did not have a copy of Archie itself we then turned to a Facebook group for internet Old-Timers self-deprecatingly called the internet old farts Club if anyone knew where to find Archie we thought they'd be here many in the group did remember the impact that Archie had but again we turned up empty-handed for an actual copy of Archie after more weeks of searching we were beginning to think this might be a wild goose chase for something that has simply been lost to time but the fact is is that little remains of the early internet even at its peak there were under 100 Archie servers worldwide and despite a thriving wear scene during the 9s the server saw software wouldn't have had much value to individual users so it's unlikely to have ever been pirated but still it seems unbelievable that there's not a copy of Archie still somewhere out there with all of our leads exhausted our last ditch effort was to try to find and contact the old administrators of Archie servers but we had nothing to lose so we started at the top of the list this ended up being an arduous weeks long process of using the Wayback machine to find Clues as to who ran and the Archie server and then investigating to see if we could find their contact information and this seems like a one in a million shot what are the chances that any of these universities or institutions would still have a backup of Archie from over two decades ago amazingly we ended up receiving a few replies the it help desk at ruter University here in the US ended up tracking down a former network administrator Dan ricean He oversaw the Rutter's Archie service in the late '90s during the period leading up to their Archie server shutting down it had seen very little usage it had been running on a spark station 20 named dogbert which was finally decommissioned sometime in 1998 prior to that though it had been the largest Archie server in the world but striking out again all the backups had been lost in more weeks went by with no leads but then we received an email reply from a systems administrator at the University of Warsaw in Poland amazingly their Archie server had been online with an old database until sometime in 2023 when it was decommissioned but more importantly they had discovered a backup of Archie files and in them the actual Archie source code and binaries we found it Against All Odds all that effort and time searching for Archie had finally paid off these were almost definitely the last copies of Archie still in existence somehow this relic of the past had been found and with image's permission we set out to resurrect [Music] Archie well here we are possibly some of the last copies of Archie still in existence the files we received consisted of four tarballs a readme and a postcript file the readme provided a list of files in their contents there are backups of their server taken in 2011 and in 2021 as well as two tarballs of the source code and one looks like it was a failed compile attempt the postcript file is a topology of Archie servers from 1995 showing how data Gathering was distributed around the world the first archive we took a look at was the source code archive in the doc directory we found an install doc which reveals that this is a beta of version 3.5 the final version of Archie and in the manual directory we found a complete latch based manual which we've converted to HTML using pandoc and we've published it in its entirety on the serialport website with a link in the description below from the docs we discovered that one of the most important differences in the later versions of Archie was usage of something called the Prospero virtual system Prospero was created in the late 80s by Clifford Newman at the University of Southern California it's a virtual file system and this version of Archie uses a modified Prospero server and protocol to efficiently handle user queries rather than the more resource intensive grep string searches of the earlier days and after some more exploring around in the other archive files we may have hit Pay Dirt we stumbled upon the dist directory in the University's backup from 2011 it looks like it contains the original distribution files for Archie with these we should be able to do a clean install so let's get started on bringing Archie back into the 21st century we first created a Kimu virtual machine to emulate spark hardware and then installed Sun OS version 5.6 as this will serve as our platform for Archie from there we simply follow the instructions from the install doc first and foremost it starts with the user requirements on the system it requires that two users are created Archie and Arch user but confusingly both need to have the same user ID which we can't seemingly do with a user ad command however we did find an option of- o which allows the user to be added with the same ID second there needs to be a user group added called Archie and both users need to belong to it once that's done we can move on to the files required for installation we see the base archive install archive as well as the specific sunos 5.4 files that we need we also have the unwrap unar and un rotate scripts as well it looks like we simply run unwrap to begin the install process that was fast but we encountered an error of re undefined variable editing the unwrap script we can see that because we're using Sun OS 5.6 there is a version mismatch we hardcode the re variable and restart the install all [Music] and now we're ready to run make except we don't have make [Music] installed after quickly doing that we can now successfully run make the install is done so we've reached our first goal but now it's time to start the configuration to get Archie configured the documentation indicates we should first set something up called the pseudo domains database this allows top level domains like UK or Au for example to be segmented into defined regions of the world to make it easier for users to search FTP sites that are close to them geographically in our case that won't matter so we'll just go with the default specified in AR domains. CF and then we run the AR domains program in the bin directory to create the database next we need to add a line to inet d.com for the AR server process and after that we add a line to Etc services for AR server as well we then need to modify the AR retrieve docf file to update the timestamps we changed these from 1994 over 30 years ago to the current date and time and then the AR update. CF file specifies which Archie servers we will do a data exchange with which Alan Mage the creator of Archie explained when there were multiple Archie servers there was a whole system of exchanging data between the RG servers so that was rgie 3.x 3.0 uh when we sort of made it commercial and and all that um and so each the world was basic basically divided up into either by domain name or well Always by domain name but domain names could either be country based or or or functional based so you could have you know an Arie server would be responsible for the all the Korean sites and it would collect all the data for the Kore Korean sites and then there was a process where it would package all of that up and other RG servers would connect to it and say give me all that data so it was sort of pre-digested data they would then have and they could then distribute that to other RG servers in our case though since this will be the only Argie server in the world we'll just remove the bunny up.com entries for now and we're finally ready to add an FTP site that we want to index to do so the docs recommend using the host manage program that's included with the installation it's a curses based program that uses emac control commands pressing tab brings up a menu and we press one to add a new site we're specifying hobbs. serialport dorg our mirror of the os2 Hobs archive that was recently shut down for the operating system we get a choice of unknown Unix BSD VMS standard or Noel great we'll just leave it on Unix BSD for now we also need to add this to the non FP catalog since this is an anonymous FTP site as opposed to the web index or another custom catalog we're then asked for a file name but it's not clear what this is does this mean the file name of the database entry or a file name on the FTP site we just enter the host name for now we Chang the status to active and then save the entry by pressing control U we weren't sure what to do next to get the FTP site updated so we consulted the Chron tab entries from the University of Warsaw installation we ran several of these commands with seemingly no activity until we saw the arch. log file entries where it complained about an empty AR update. CF which is the file that we removed the bunny up.com lines from oops we restored that and tried again running AR retrieve with the dasl option does produce something in arch. log we see that there is a connection to Local Host but other than that nothing else so we then used truss a debugger for sun OS to see if we can get any more information from AR retrieve we do see that it is opening and creating files and it does appear that it is updating the date and time in AR retrieve docf as this file is now set to April 24th we then find out we can use the DF option to specify a host name to manually update and finally some progress we have a header file written out to the disk which is used by the next process to actually connect to the FTP site and then we stumbled upon a sentence in the docs that states that we can use the- V flag on most commands to get a more verbose output so we run AR control- r-v which runs the retrieve a non FP program and oops we can see that it's trying to download a file named hobbs. zero.org this is what the file name field in host manage must control so going back into host manage we can update the entry and remove that value running AR retrieve again and then AR control we have what looks like success Archie connected to the FTP and retrieved a recursive listing the next step in the process is to parse the output generated by AR control-r so now we use the- p and- v options and encounter our next error we're seeing more file not found errors manually running the command at airdon reveals the problem we don't have Pearl installed after getting that installed we figured out we have to rerun the indexing process from the beginning and now we run AR control with- U to update the database and once that is completed we see now that there are string files indicating that we should have a searchable database now Archie comes with several built-in clients including a telnet client we fire that one up and change our search type from exact to sub to allow paral matches from our query but we get an ARP timeout it turns out that the Prospero server needs to be running for the telnet client to work so after starting that and trying again we now get a super helpful error message we honestly weren't really sure we to turn next so we decide to just give it another try and lo and behold it works our new Archie server is now fully up and running with a searchable database we can even use remote clients to query the server and after indexing a few more FTP sites we've got plenty of data to be able to search and finally we can say that Archie is back we've achieved our goal of bringing Archie back but Archie was meant for internet users and that's where you come in we've opened our Archie service to the public at arch. sereral port. org the interface is based on arlex the web Gateway we talked about earlier and we were surprised to see that Archie could actually be somewhat useful in the modern day the listings are hyperlink so if you have an FTP client set up for your browser it enables pretty quick and easy file downloads without needing to wait through a bunch of web pages to find what you're looking for and with Archie now up and running for the 2st century our journey is now complete when we started this project we had no idea if we were actually going to be able to find Archie so we owe everyone who helped us out along the way a huge thank you and especially to Mike Schwarz and Alan mage of course and thank you for coming along with us and we'll see you next time [Music]