posted on Jun, 28 2020 @ 05:50 PM
Just to follow up on my previous post, I've completed the basic script idea and have hit the wall on diminishing returns between time spent and a
'good enough' product. It doesn't work for really above top secret posts, but I'm not too bothered by that. If someone points me some threads there
which some people think are genuinely
worth preserving I'll look into adding credentials into the script which would allow access to that
forum, but I'm not really holding my breath.
For threads that aren't huge, the pdf archival methods which have been discussed above and on other pages are certainly a better option than this
script. They provide a much more complete archive with working direct reply links and capture all of the visual content on the page. But for the
gigantic threads that overwhelm pdf memory limits this might be the way to go.
Since no mods or admins have chimed in to my knowledge, I've looked through the terms and conditions and the relevant section is here, emphasis
6) Use Of Content; Copyright. All content created by Users of TAN website are governed by an "Attribution - NonCommercial - NoDerivs 3.0 Unported"
Creative Commons Deed (hereinafter "Creative Commons License"). Users can share, distribute, copy, and transmit the work Users find on TAN websites
provided Users provide proper attribution that includes the name of the Website User, the name of the Website, and a link to the content thread. These
usage rights do not apply to any commercial use of the content found on TAN websites, and such use is strictly prohibited without prior written
authorization from TAN. These usage rights also do not apply to derivative works, and such use is strictly prohibited without prior written
authorization from TAN. Each post of each thread contains a "copyright" link that clearly specifies the method in which the material may be shared,
distributed, modified, or used.
You will not copy, distribute, display, or publicly perform material from the Websites to Post on other websites or quote in offline research in a
manner that does not comply with the Creative Commons License that specifies the author's member name as a member of AboveTopSecret.com, includes
The Above Network, LLC and the member as owners of the content, provides the title of the thread, and a full URL to the board thread.
I'm relatively certain that the intended goal of this project falls within the allowed scope of use, so I may share the little script with other
prominent forum contributors who have legitimate need of it, either to preserve their own content or because of their knowledge of important and
notable threads. Those folks know who they are and can contact me for access.
But I won't be sharing it with the average ats user (oh the hypocrisy). There's just too much risk of inadvertently DDOS'ing the site, either through
too many people archiving at once or because of some simple modifications to the script can easily overwhelm the server.
Anyway, I really hope this whole project ends up being pointless and ATS lives on.
edit: Oh and one last thing. This script only works in the command line, so anyone interested needs to be technically savvy enough to install wget
successfully in their windows environment or have access to a unix environment. It will require slight modifications to run in a windows environment,
and I may add those if requested. Though I imagine anyone who can run the script can figure out how to safely make those changes as well.
on 28-6-2020 by framedragged because: (no reason given)