My home lab has a mild amount of complexity and I’d like practice some good habits about documenting it. Stuff like, what each system does, the OS, any notable software installed and, most importantly, any documentation around configuration or troubleshooting.
i.e. I have an internal SMTP relay that uses a letsencrypt SSL cert that I need to use the DNS challenge to renew. I’ve got the steps around that sitting in a Google Doc. I’ve got a couple more google docs like that.
I don’t want to get super complicated but I’d like something a bit more structured than a folder full of google docs. I’d also like to pull it in-house.
Thanks
Edit: I appreciate all the feedback I’ve gotten on this post so far. There have been a lot of tools suggested and some great discussion about methods. This will probably be my weekend now.
Thoughts and Prayers.
The real enterprise solution.
Ibrun my own gitlab instance where i have a repository with all my configs, scripts etc.
As well as a wiki that contains the admin guide with (hopefully) all the relevant infos in form of text and PlantUML graphs.The nice thing is… You can just ‘code’ the diagrams and use a PlantUML instance to render the graphs live
I’m using netbox, and I’m in the process of info dumping my brain to a media wiki. I may add ansible into the mix in the near future.
My lab is a bit large and complex and I’m currently in the process of trying to train help to run it, from the pool of people that make use of it. They know how the front end works, a few of them need to learn the back end.
I’ve been using Obsidian for a lot of other purposes for a couple years now, so I was comfortable adding my documentation into my existing vault there. I made a couple templates that I fill out for any hardware/software/networking equipment.
Since the app’s selling point is storing all your notes in plain text I wouldn’t put anything security-related in there without some encrypted container. I use KeePass for that part, and keep the file it generates in the same folder as Obsidian so I can link to it within notes. Click the link in the note, KeePass opens the vault and asks for its password.
Mine is, er, self-documenting, and my partner has instructions, in the event that I die, to plug the wifi router into the modem and unplug all the other crap. The router has a sticker saying ROUTER and the modem has a sticker saying MODEM.
As I move to more self hosting, it’s becoming more and more important to create a “what to do if I die” procedure for my wife (or even children) to follow. I mean it’s not big deal if the plex server goes down and doesn’t come back up, I’m thinking more along the lines of all of our photos, important documents, password manager, those type of things. I have 3 - 2 - 1 backups for the important stuff and have tested them, but that means nothing to my wife if I wasn’t around to get that stuff back if something happened… I wonder some days if I should document it all and put a print out with a step by step guide on how to get everything back that a semi tech savvy person could follow.
Yes, I really need to do this too. It’s a fairly daunting task because my significant other is not at all comfortable with even slightly complicated operations on a computer. I need to figure out a way to make a really easily accessible backup of everything.
Self documenting, eh? I may be familiar with the same process.
You just follow all the wires and read all the config files. Easy!
Joplin synced to webdav/nexcloud.
I use Ansible, Docker, and Emacs OrgMode files committed to Git. Diagrams are a mix of Miro and Graphviz. There’s also a few markdowns in there too. Joplin is used for rough notes only.
I use draw.io for diagrams. Netbox to keep track of devices, IP addresses, and cables. MediaWiki for how to articles. Both Netbox and MediaWiki live on a VM both at home and offsite and they sync nightly.
For me, making a graph is the best documentation, everything else is in config files.
If I really do need to takes notes, It’s currently in notion but I’m hoping soon I’ll be able to self-host AnyType (https://github.com/orgs/anyproto/discussions/17)
I use WikiJS for documentation. Simple, powerful and has a lot of features
+1 for WikiJS. As a bonus you can have WikiJS back itself up to plain text MarkDown files, so if things explode you can always just read those from wherever.
Another great feature I use is to have WikiJS back itself up into git. If I am going to a place with no internet access I can do a quick git pull and have a complete copy of my wiki including files on my laptop.
I use the wonderful https://draw.io to sketch up my homelab and which device hosting what service. More fun when it’s vidualized the way I want it 😊
When I had home lab I used to use racktables - https://www.racktables.org/
but I’m not sure if this is what you’re looking for.
The only thing I save in Google Drive are my notes just in case of disaster.
Frankly the only thing I’d save in Google Docs are encrypted archives. Otherwise they’ll profile the documents to send ads to you. But it is a good back up in case lightning strikes your home or something.
I don’t save all my documents. Just my self-hosting, servers infraestructure notes. I don’t want to have the recovery intructions in the same machine I’m recovering
I’m not very good at deciding on where to document things, so I have a mix of BookStack, Dokuwiki, and Obsidian currently.
I really like Dokuwiki but I like the UI/UX of BookStack better so I’m working on a plugin to sync bookstack and obsidian. I’ll probably get rid of Dokuwiki after that.
The main reason for syncing with obsidian is that I want documentation that isn’t stored on the thing it’s about, in case my servers completely die.
In another thread, someone reminded me that TiddlyWiki still exists, it’s also a pretty cool little tool.
I deploy as much as I possibly can via Ansible. Then the Ansible code serves as the documentation. I also keep the underlying OS the same on all machines to avoid different OS conventions. All my machines run Debian. The few things I cannot express in Ansible, such as network topology, I draw a diagram for in draw.io, but that’s it.
Also, why not automate the certificate renewal with certbot? I have two reverse proxies and they renew their certificates themselves.
This is the way
psssst. ansible is red hat.
red hat bad.
What alternative to you suggest?
i m all good with selfhosted wiki
I meant to replace Ansible automation. Pointing out it’s RH is all well and good, but what’s the alternative?
My reverse proxy can do automated renewal just fine. The SMTP relay requires a DNS challenge that is manual.
Why not have the reverse proxy also do renewal for the SMTP relay certificate and just rsync it to the relay? For a while I had one of my proxies do all the renewals and the other would rsync it.
It certainly wouldn’t be because I’ve been doing it this way for so long that it never occurred to me. Nope. Certainly not that.
In fairness, I very recently switched from a cobbled together apache web server/rev proxy config I’ve been carrying along in some form for well over a decade (I remember converting the config to 2.4), to an NPM container. I had some initial trouble switching my certs over to NPM and haven’t revisited that yet.
I’m in the middle of a major overhaul of my tech stack. Fixing certs is on my short list.
Thanks for pointing out where I was stuck in my ways.
