Troubleshooting Why My Nextcloud Server Stopped Working
Video Title: Troubleshooting Why My Nextcloud Server Stopped Working
I ran into an issue this morning with one of my web servers I’ve got a nexcloud server that uh is is actually sitting on an abon server that I installed nexcloud via the snap package because on a boon server it really makes sense to use snaps to install things
Like nexcloud for example because the snap package of course has all the dependencies it’s got the entire lamp stack you know Apache and my seel and all of that PHP it’s got all of that bundled up in the snap package and that’s one of the really beautiful things about snap packages is actually
Snaps on the server are fantastic so all I did was spun up in a boon2 server a couple of years ago did a snap installed nexcloud and I’ve had this nexcloud server running for about 2 years and I use it really as a file syncing service
To sync all the files on my workstation here at the office with my home computer as well as my laptops if I carry them out of town on a vacation or something or a trip it really makes things convenient because I can have some of my work documents like show notes or even
Video files things like that I can edit them on one machine and those changes are synced to all of my machines via the nextcloud server that I’ve been running for again the last couple of years but today I got up my next Cloud Server was not working it was down so naturally I
Had to spend a couple of hours this morning investigating the issue and it’s one of those things you know troubleshooting Linux on the desktop and troubleshooting Linux on the server are very similar because typically all you do is you go to the command line and if
You know some of the basic gnu coreutils you can quickly investigate the issue you can find what the problem is on that system and correct it let me show you some of the steps I went through this morning to get my next Cloud Server back up and running so let me switch over
Here to my terminal here on my Linux desktop and the first thing I had to do was I needed to SSH into that remote web server right because if I’m going to investigate the problem on that server I need to switch over to that server via
SSH so you know you SSH like this you run SSH space and then the name of the user for example Derek if the username is Derek that I want to log in as or roote if I can log in as root and then some IP address something like I don’t
Know I’m just going to make up some IP address but it’ll look like this right I’ll SSH and then name a user at IP address hit enter and then of course I’ve got to know the root password for that particular machine I’m sshing into but of course I knew the password and
Then once you log in you know now your command prompt changes and you’re now on that remote server all the commands you enter at that point will take effect on the server not your local machine so the First Command I entered was the DF command because I’ve encountered problems on web servers before
Especially when you’re hosting things that collect large amounts of data in this case This nextcloud Server I host a rather large amount of files on this thing I ran the DF command DF tells you the uh the disc space being used on your system essentially if I DF with no other
Arguments it’s going to list all of your block devices all your Loop devices and tell you how much space is being used you can see use percentage now of course this is my local machine this is not the remote remot server but just imagine we were on my remote server the nextcloud
Server here and imagine this is the main Drive Inlet server and you can see that’s the root file system and you can see I have used up 57% of the root file system here on my local machine well when I ran the DF command on my nextcloud server the usage percentage
Was actually 100% so that is not good when you have 100% of the storage being used on your system obviously that’s a problem that’s going to cause a pretty much a denial of service the server is no longer going to work right whenever I went to the web page for my next Cloud
You know nothing happened it’s because essentially that machine is essentially Frozen at that point it’s it was actually running but it couldn’t do anything because there just wasn’t any free space available for it to do any kind of rights to the disc so once I
Knew that the drive was full I knew I needed to actually delete the the directories the files I needed to create some space and what I wanted to do is I wanted to find the problem directories on that system that were taking up so much space because my nexcloud server
Has a 80 gig Drive SSD drive and I knew I wasn’t storing 80 gigs of stuff in my next Cloud directory here on my workstation I’ve only got about 15 or 16 G gabt of actual data that I’m sinking to my next Cloud Server and then the auntu server operating system probably
Takes up another 10 or 15 gigs of space so really this 80 gig Drive should have been fine because I I probably should only be using about 30 to 35 gigs of space on that 80 gig SSD why is the drive full something was was not right
And what I did after that now that I knew the drive was full I ran the the DU command du for disk usage what this does if I do it on my root file system it’s going to take forever I’m going to cancel it but what it does is it scans
Every directory in the root file system and tells you how large each directory is now that’s really nice because what you can do with the magic of piping you could do something like du and you could give du the- a flag for all meaning all files and directories on the system cuz
Sometimes you could have some very large just single files on the system and you want those listed as well and then you give it a path if you don’t want to do the root file system you would do something like slome SL DT or you know
On my nextcloud server I could do SLR SL snap I think it’s the pass SL uh nextcloud or whatever it happens to be but here on my system to show you this in action I’ll do slome D and I’ll pipe this du command into sort because I
Don’t want just a random bunch of gibberish outputed because there could be hundreds of thousands of directories in this list right and I want you to sort it and I want you to give me the biggest directories I’m not interested in the small directories at all I want
The biggest so what I’m going to do is I’m going to do a dash in so this is a numeric sort so give me uh essentially a numerically sorted list- R so give me the biggest directories first because by default it would do it the opposite right it would start with the smaller
Directories in the list but I actually want the biggest ones first and then I’m also going to pipe this into head because I don’t want every single line of output in this I really only want uh I would say the 20 largest directories on the system and if I run this it’s
Going to take a minute because it’s going to scan my entire home directory depending on what directory you run this on it could take a minute and I’m getting some permission denied errors here because I do have some root protected files in my home directory but
I could have of course used sudu or simply logged in as rout to to run this but here you have it the output of the 20 largest directories in my home directory including the top level right which is the home directory itself but you can see my music directory has 109
Gab of space taken within it and within it uh my music/ classical directory is 85 gigs of that you can also see my virtual box virtual machines take up a ton of space in my home directory the largest VM is my Windows 10 VM you can
See my trash directory has a bunch of stuff in it as well the videos directory of course on my system is typically one of the largest ones because video files are very large those are the files that I’m recording like right now making this video and of course the steam directory
Is rather large so this gives me an idea of some the space that’s being taken up here on my local machine now when I ran this on my nextcloud server the uh directory that was the biggest one and it didn’t make any sense was uh this directory named this /var snap nexcloud
Yada yada yada SL user SL filesor trashbin so there is a trash bin in nexcloud and I didn’t realize this what happens when you move files delete files on a nexcloud server well I guess it sends it to a trash bin and this trash bin directory had about 48 gigs of stuff
In it so this 80 gig SSD drive on This auntu Server you know well over half of it was stored as trash files in this trash bin so that was actually what had essentially caused my nextcloud server to cease to exist right to cease to work so after I sshed into my nextcloud
Server obviously I did a r Mr RF to remove this directory actually I didn’t remove the directory I actually cded inside the directory and I did a RM of all the files in that directory and then once I deleted those files you know I I rebooted the machine just to be on the
Safe side and then you know I’ve got my nextcloud back up and running I’ve got the little Cy trait icon here for nextcloud you can see my last sync was successful so my files on this computer my workstation here at the office and my home computer everything is back in sync
Because honestly without that nextcloud server it would make my life a living hell having to sync so many files from various computers that I use and some of these files are rather large especially once you start dealing with audio files and especially video files you know then
If I didn’t use something like a nexcloud server to sync these machines I’d probably have to use like physical devices like external USB hard drives and things like that to move some of these video files arounds and and things like that so I’m really happy that I was
Able to solve this problem and it’s one of those things I like showcasing some of this kind of real world troubleshooting on camera because I know I’m not the only one that runs into these problems and sometimes especially if you’re kind of new to Linux new to
How some of these command line tools like DF du sort you know all these good new core yous that I sometimes show on camera you know this is the reason you need to know these things because they do come in handy and if you’re unsure on
How to use some of these commands like DF du sword and some of the other G new core utils check out my commandline playlist I’ve got a playlist that’s got probably 50 60 maybe even more videos than that in it where I have made videos on pretty much most of the really
Important command line utilities that everyone should learn if they’re using Linux now before I go I need to thank a few special people I need to thank the producers of this episode Gabe James Matt Paul web armor Dragon Commander angry George Lee methos Nate Aran Paul Peace Arch Andor realities foress red
Profit Roland Soul tools Dev waru and abtu and Willie these guys they’re my high Ste patrons over on patreon without these guys this quick episode about me having to figure out why my disc was full on my next Cloud Server it wouldn’t have been possible the show is also
Brought to you by each and every one of these fine ladies and gentlemen all these names you’re seeing on the screen right now these are all my supporters over on patreon if you like my work want to see more videos about Linux and free and open source software please help
Support my work through patreon subscribe to Dro tube over on patreon peace guys
Video Keywords: Operating Systems & Servers
-
Sale!
Wireless WIFI Repeater Extender Amplifier Booster 300Mbps
$29.99$14.99 Add to cartWireless WIFI Repeater Extender Amplifier Booster 300Mbps
Categories: Electronics, Wi-Fi Router, Wireless Wi-Fi Extender Tags: 300Mbps, 802.11N, Amplifier, Booster, Extender, mobile wi-fi booster, Remote, WIFI, Wireless, Wireless WIFI, Wireless WIFI Repeater, Wireless WIFI Repeater Extender, Wireless WIFI Repeater Extender Amplifier, Wireless WIFI Repeater Extender Amplifier Booster, Wireless WIFI Repeater Extender Amplifier Booster 300Mbps$29.99$14.99 -
Sale!
Full RGB Light Design Gaming Headset Headphones with Mic
$24.99$14.99 Add to cartFull RGB Light Design Gaming Headset Headphones with Mic
Categories: Electronics, Gaming, Gaming Headsets Tags: Design, Full, Full RGB Light Design Gaming Headset, Full RGB Light Design Gaming Headset Headphones, Full RGB Light Design Gaming Headset Headphones with Mic, Gamer, Gaming, Gaming Headset Headphones, gaming headset wireless, Headphone, Headphones, Headset, Light, Mic, Package, RGB$24.99$14.99 -
Sale!
Wireless BlueTooth Multi-Device Keyboard Mouse Combo
$39.99$19.99 Add to cartWireless BlueTooth Multi-Device Keyboard Mouse Combo
Categories: Electronics, Gaming, Gaming Keyboards, Keyboard Mouse Combos Tags: Combo, Keyboard, keyboard mouse combos, Mouse, MultiDevice, Set, WireKeyboard Mouse Combo, Wireless, Wireless BlueTooth Keyboard Mouse Combo, Wireless BlueTooth Keyboard Mouse Combos, Wireless BlueTooth Multi-Device Keyboard Mouse Combo, Wireless BlueTooth Multi-Device Keyboard Mouse Combos$39.99$19.99 -
Sale!
High Back Leather Executive Adjustable Swivel Gaming Chair with Headrest and Lumbar
$199.99$139.99 Add to cartHigh Back Leather Executive Adjustable Swivel Gaming Chair with Headrest and Lumbar
Categories: Gaming, Gaming Chairs Tags: Adjustable, Chair, computer chairs, Desk, Executive, Gaming, Girl, Headrest, High, High Back Leather Executive Adjustable Swivel Gaming Chair, High Back Leather Executive Adjustable Swivel Gaming Chair with Headrest, High Back Leather Executive Adjustable Swivel Gaming Chair with Headrest and Lumbar, High Back Leather Executive Adjustable Swivel Gaming Chairs, Leather, Lumbar, Office, Racing, Swivel$199.99$139.99 -
Sale!
Professional LED Light Wired Gaming Headphones with Noise Cancelling Microphone
$29.99$19.99 Select optionsProfessional LED Light Wired Gaming Headphones with Noise Cancelling Microphone
SKU: N/A Categories: Electronics, Gaming, Gaming Headsets Tags: Cancelling, Gaming, Gaming Headphones with Noise Cancelling Microphone, gaming headset, Headphones, Headset, LED, Light, Mic, Microphone, Noise, Professional, Professional LED Light Wired Gaming Headphones, Professional LED Light Wired Gaming Headphones with Noise Cancelling Microphone, Wired, Wired Gaming Headphones, Wired Gaming Headphones with Noise Cancelling Microphone$29.99$19.99 -
Sale!
Gaming Desk with LED Lights USB Power Outlets and Charging Ports
$349.99$249.99 Select optionsGaming Desk with LED Lights USB Power Outlets and Charging Ports
SKU: N/A Categories: Computer Desk, Gaming, Gaming Desk Tags: and Charging Ports, Charging, Desk, Desks, Gaming, gaming desk with led lights, Gaming Desks with LED Lights, Home, LED, Lights, Monitor, Office, Outlets, Port, Power, Room, Stand, USB, USB Power Outlets, White, Workstation$349.99$249.99 -
Sale!
Wired Mixed Backlit Anti-Ghosting Gaming Keyboard
$99.99$79.99 Add to cartWired Mixed Backlit Anti-Ghosting Gaming Keyboard
Categories: Electronics, Gaming, Gaming Keyboards Tags: Antighosting, Backlit, Blue, brown, Gaming, Gaming Keyboard, gaming keyboards, gaming keyboards and mouse, Keyboard, Laptop, Switch, Wired, Wired Mixed Backlit Anti-Ghosting Gaming Keyboard, Wired Mixed Backlit Anti-Ghosting Gaming Keyboards, Wired Mixed Backlit Gaming Keyboard$99.99$79.99 -
Sale!
Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset
$119.99$59.99 Add to cartWireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset
Categories: Electronics, Gaming, Gaming Headsets Tags: 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset, ANC, Audio, Bluetooth, Cancellation, Ear, Earphone, gaming headset, Headphones, Headset, Hi-Res Over the Ear Headphones Headset, HiRes, Noise, Wireless, Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Headphones, Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset, Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headsets$119.99$59.99 -
Sale!
Wired Sports Gaming Headset Earbuds with Microphone
$19.99$9.99 Select optionsWired Sports Gaming Headset Earbuds with Microphone
SKU: N/A Categories: Gaming, Gaming Headsets Tags: Accessories, Earbud, Earphone, Earphones, Gaming, gaming headset with microphone, Headphones, Headset, IOS, Microphone, Sports, Wired, Wired Sports Gaming Headset Earbuds, Wired Sports Gaming Headset Earbuds with Microphone, Wired Sports Headset Earbuds$19.99$9.99 -
Sale!
150W Universal Multi USB Fast Charger 16 Port MAX Charging Station
$49.99$29.99 Add to cart150W Universal Multi USB Fast Charger 16 Port MAX Charging Station
Categories: Charging Stations, Electronics Tags: 150W, 150W Charging Station, 150W Universal Multi USB Charging Station, 150W Universal Multi USB Fast Charger 16 Port MAX Charging Station, 150W Universal Multi USB Fast Charger 16 Port MAX Charging Stations, 150W Universal Multi USB MAX Charging Station, 16 Port MAX Charging Station, 3.5A, Charger, Charging, Fast, laptop charging stations, Max, Multi, Port, Stand, Station, Universal, USB$49.99$29.99
ncdu ftw
DT. Is there any chance you could creat a video on what to do when your encryption goes wrong..
Doom threw up a request for a passphrase for symmetric encryption, I mistook it for a request to re-enter my password for a remote server I was trying to connect to.. from then on the passphrase was required, it didn't cause any real issues. It may even have made connecting to the SMB share easier.
However, an update for Ubuntu 23.10, came around, so I did it. Of course this led to Doon needing updated. So I did it.
But somewhere, along the upgrades, the encryption folder system broke and has effectively locked me out of Doom, as the first thing it wants is the encryption passphrase, but doesn't recognise it. I am effectively locked out of Doom. A full re-install of Doom has not fixed it ( I had hoped it would create a new passphrase file) it doesn't. It seems to be external. Held somewhere on the Ubuntu system, but of course the relevant files on it are 'encrypted', and I have no idea what to do to clear it and the problem.. which may break other aspects of my system.
Help me Obi Wan Kinobe, your my only hope…
Because of DT, Chris Titus, The Linux Cast, Switched to Linux and several other Linux YouTubers, I finally switched to Arco Linux and deleted Windows completely. I still hate the terminal but I’m slowly learning it. I set up my machine through GUI’s to include using cockpit to built my RAID6 array for me, Gnome Disks to auto mount, change ownership etc. I bought and set up a raspberry pi with pi hole, unbound, no ip duc and Tailscale.
I also set up a vanilla arch machine to learn on but for now I used the calamares installer – don’t cut my head off for this, I will install Arch “the arch way” when I have a better understanding of it.
How do you monitor your services?
Sweet vid DT.
Why does nextcloud fail to connect if the disk is full?
Great video, very informative! Please make more.
Who knew!!
The first thing I would have checked is syslog.
Thanks DT! Good vid and info!!
When I'm annoyed by a flurry of error messages, I like to pipe them to /dev/null. Just add "2>/dev/null" to the offending command in the command-line. That 2> thing is a handy thing to learn.
A colleague of mine mentioned somebody filled the file system to such an extend, that my colleague had trouble even loggining in. I still wonder how he managed to clean up that mess 😀
Thanks for this video, much too rarely do you see videos or guides about troubleshooting issues. Especially for me as a beginner to intermediate Linux user, who does some server and VM stuff but isn't deep in IT, I very often don't know where to start if I'm experiencing a problem. It's very helpful to see some basic tools used to actually diagnose what's wrong from the beginning. Most of the time guides are very specific on the topic of one specific problem but I just don't find them because I don't get to what the underlying issue actually is. So great job, DT, thank you very much! Glad to be a subscriber!
How about ncdu?
Syncthing opensource too
I'd like to recommend "ncdu" utility. It is pseudo-graphical version of "du" that scans all directories and lets you navigate through them interactively. I personally use it all the time, as I store large files and often run out of space.
I've made myself handy alias ncdux='sudo ncdu -x –confirm-quit'
-x ignores mounted dirs, as usually you don't care about them when looking for wasted space on your drive
and –confirm-quit doesn't let you scrap all the scanning because of accidental "q" or "^C"
Maybe it is a good time to review some monitoring solutions, that would notify you in advance when something like this happens
magicraminstaller 💀
My next cloud runs on the same 120GB SSD as my torrent folder as I only use it for syncing and archive files as they pile up.
One day I added a big torrent with many Linux iso and the next day my nextcloud wouldn't connect. I just missed the fact that I didn't have enough space for the torrent
My man made a 10+ min video trying to justify why he wasted his morning by not clearing his trash bin. Lesson learned.
first