Rec
⌨️
learn to scrape the internet
you people seem cool so i try to avoid too much nerd/code/tech talk but here we go anyway. 🤓
one of the most useful skills i have acquired as a coder is the ability to steal things from the internet. there is so much stuff on the web that you ostensibly are supposed to pay for, but with a little coding know-how you can scrape the relevant data, download it to your computer, keep it forever and never pay a dime for it. Or maybe pay for it once but be able to cancel your ongoing subscription to it.
I have used this skill to steal / borrow / archive: - workout videos from a paid yoga app - cocktail recipes - food recipes - music - library books - instructional materials - graphical source material for making art
Depending on what you’re doing, this can require more or less technical skill, but likely less than you expect. this is also a thing that chatgpt is very good at walking you through, if you ask it for help. it feels especially rewarding to learn a new skill in service of saving yourself money, too. Even if the content in question is free, it’s nice to know you’ll have it for eternity even if a provider goes out of business, and it enables you to search/organize/manage the content however you like (with no ads, paywalls, etc).
it’s a useful exercise in digital literacy that demystifies a little bit of the technical world that surrounds us, and it always gives me a thrilling little frisson of “civil disobedience” or at least naughtiness.
recommendation image
image
@heumann
Apr 28, 2024
14
see likes

Comments

Make an account to reply.
image
@ethansawyer
i like the idea of AI working for me, instead of the other way around
May 22, 2024
2
image
@sandym
Wait - have you gone through the process of building your own GPT bots? The data scrubbing portion is what's keeping me farthest from it but I'm interested in learning more. 🥸
May 21, 2024
1
image
@heumann
sandym I have made my own GPTs actually! Limited utility so far. but as far as the data scraping..… let vanilla chat gpt show you how to do this too. I had it write me a script this morning that was like “find all the files in this directory, read the text from each one, and add it to a new single text file” and then just popped that into my custom GPT’s file upload thing, seemed to work well.
May 21, 2024
2
image
@sandym
heumann I’ll give it a try! 👉👈 will give update on how it goes
May 22, 2024
1
image
@taterhole
I would love to see nerd posts from you by the way because I love generative AI on a personal level for creative use but I’ve also been dipping my toes into artificial intelligence market research at work 🤓
Apr 29, 2024
1
image
@heumann
taterhole 🥲 ok thank u for encouragement more nerd posts on the way!!!
Apr 29, 2024
1
image
@taterhole
heumann YAYYY please!!!! 🙌
Apr 29, 2024
1
image
@taterhole
Omfg this is sick thank you for sharing!!!!!
Apr 29, 2024
1
image
@taterhole
taterhole the ChatGPT script example lol. I love doing stuff like this
Apr 29, 2024
1
image
@thewarblerette
Dont be scared to share what u want on here! This is what the website is for!!
Apr 28, 2024
2
image
@heumann
thewarblerette 🩷 thank u, i appreciate it! I was mostly joking but also i would be genuinely sad if this place became overrun by people arguing about programming languages and front-end frameworks and blockchains and shit like all my other social media. just trying to be a responsible maintainer of the vibe
Apr 28, 2024
2
image
@thewarblerette
heumann LMAOOOO that's okay 😭 there's like 20k+ people on this site, the vibes are gonna be weird sometimes, it's bound to happen
Apr 28, 2024

Related Recs

Rec
⌨️
making websites :)
been on neocities experimenting a little and 1) it's so fun seeing other people's sites???? they're so creative and cool and inspire me a ton
2) it's such a fun way to spend 3 hours uninterrupted just Making something. looking up yt tutorials and kicking that index.html file. cus sometimes u just need to make stuff man
3) it's a kickass long term project to occupy ur freetime, teaches u a useful skill, and allows u to showcase ur work especially if ur a creative?
4) it's free???? to some extent at least -if ur using a free host (like neocities) and don't need a ton of memory space
overall lovely thing to get engrossed in. would recommend :)
image
@bimbofication
Feb 5, 2025
3
Rec
👩
creating ur own corners of the internet
let’s face it social media is dead.
create your own corners of the internet! to code is to world build.
i coded and designed my own blog two years ago (it’s linked if you want to take a look at it)
it’s full of my thoughts, music, art, literally everything i love.
i lowkey abandoned it (school was beating my ass but we move)
i’m coming back to it because doesn’t require that i adhere to algorithms or trends.
it’s authentic. it’s feels true to me. it feels like home
even if u don’t code, start off with pinterest, are.na, substack
i feel like the internet can be used to connect us maybe just not in traditional ways like social media.
image
@digitaldiva
Nov 5, 2024
51
Rec
⌨️
boolean search functions
i still use them :) i felt so changed when i learned about them in 8th grade. “-” when i wanted to omit something from the results? blew my mind. google has been going steadily downhill yeah but like i think if people knew how to use these techniques and they still taught decent tech literacy in schools, they wouldn’t turn to more unreliable and unethical things like chatgpt as much. (google is by no means an ethical company to me, don't get it twisted).
image
@deardoveswings
Aug 30, 2024
6

Top Recs from @heumann

Rec
🐈
im watching my cat throw up on my yoga mat
even tho i very specifically asked her not to do that
image
@heumann
Mar 13, 2025
20
Rec
🏢
The arbitrary intimacy of apartment buildings
i love hearing that girl sing her heart out in the shower through the bathroom vent. or the guy hocking insane loogies all the time. the alternating smells of delicious homemade chinese food or something totally burning in the oven… am always wondering if its the same kitchen!
image
@heumann
Feb 14, 2024
19
Rec
🗣
Stubbornly continuing to pronounce a word wrong in your head
i still read hyperbole as “hyper-bowl”
image
@heumann
Mar 18, 2024
19