Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: Pi - SD card wear with mysql employment  (Read 6123 times)

roswellgrey

  • Member
  • **
  • Posts: 73
Pi - SD card wear with mysql employment
« on: June 22, 2016, 09:32:33 AM »

<snip>

Long term soak test seems the only realistic way forward to see if a very early card failure happens :)

One of the more interesting things to come out of that aged thread were concerns about long term write wear on a Pi's SD card (in my case a cheapo blue Sandisk) and whether continual usage of mysql and other regular card writes would, in effect,  rapidly "wear out" the card.

Well, 18 months on, I thought some might be interested in an update.

The mysql database now has > 350,000 rows (a mixture of normal data and some quite heavy blobs) & I deliberately left on a horrible amount of debug file logging to maximise the number of writes to the card.

Outcome (so far): The SD card has behaved itself, and has exhibited no problems whatsoever.

How long will it last ? Who knows .... only time will tell ... if it ever fails, I will post an update.
Logged
Billion 8800NL

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: Pi - SD card wear with mysql employment
« Reply #1 on: June 22, 2016, 04:44:41 PM »

Thank you for that update, it is useful to know.  :)
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Re: Pi - SD card wear with mysql employment
« Reply #2 on: September 04, 2016, 04:11:03 AM »

Very worrying, the whole idea of flash wear. After all one of the main attractions is to get higher reliability compared with mechanical disks.

Does anyone know if such a thing as higher durability flash is available from some quality suppliers?

Who makes the fastest high quality flash drives in the categories of write speed and read speed ?
Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: Pi - SD card wear with mysql employment
« Reply #3 on: September 04, 2016, 07:44:10 AM »

Flash wear is certainly an issue, but it's less of an issue with newer flash drives. Several people are using a Raspberry Pi for running DSLstats and uploading to MDWS, and this involves several reads and writes every minute while processing and uploading data from the modem. The flash drives won't last for ever working like this, but they seem to be holding up pretty well.
Logged
  Eric

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Re: Pi - SD card wear with mysql employment
« Reply #4 on: September 04, 2016, 08:05:59 AM »

Aside: On a Linux box such as most Raspberry Pis, is there an alternative to flogging flash drives with frequent writes? Is it possible to have a ramdisk? Although I'm sure physical RAM will be in extremely short supply, so stealing it might hurt system performance badly. A very modest one might give performance gains for temp files, although that might not suit this particular use-case for several reasons. Also if Linux does flash file system caching in using RAM with lazy write-back then the performance gain might be non-existent.

Btw does Linux on a Raspberry Pi do virtual memory with a pagefile on flash? Fantastic to have a non-existent ‘seek time’ for random access, the killer with pagefiles, but evil in terms of wear because of the huge number of writes per second.
Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: Pi - SD card wear with mysql employment
« Reply #5 on: September 04, 2016, 10:41:22 AM »

By default the RPi has a swapfile of 100 MB set up in /var/swap. The later versions of the RPi have 1 GB of memory, and the way the Linux memory manager works, the swapfile will not be used until the memory is fully occupied with programs and data. The swapfile can be moved elsewhere, such as a USB hard disk if you have one. See http://raspberrypimaker.com/adding-swap-to-the-raspberrypi/

A ramdisk is certainly possible, and a fairly small one would be sufficient for DSLstats to hold temporary data during sample processing and MDWS uploading. But I don't believe this could be done by DSLstats running with ordinary user privilege, so it would need a knowledgeable user as well as a special version of DSLstats for the RPi only. I'm afraid I don't think this is practicable.
Logged
  Eric

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Re: Pi - SD card wear with mysql employment
« Reply #6 on: September 04, 2016, 10:48:30 AM »

could a ramdisk be mounted as just another node in the tree and the app work off a TMP or TEMP environment variable to know which directory to create certain files in ?
Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: Pi - SD card wear with mysql employment
« Reply #7 on: September 04, 2016, 10:55:36 AM »

A ramdisk would have to be mounted somewhere in the file system. But it would still require a knowledgeable user and an appropriately modified program to make use of it.
Logged
  Eric

ejs

  • Kitizen
  • ****
  • Posts: 2078
Re: Pi - SD card wear with mysql employment
« Reply #8 on: September 04, 2016, 01:41:12 PM »

I run Fedora on my Pi 2, I thought it mounts /tmp as tmpfs (an in RAM filesystem, but without having to set up a RAM disk), but unlike my desktop, it doesn't.
Logged

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 7382
  • VM Gig1 - AAISP L2TP
Re: Pi - SD card wear with mysql employment
« Reply #9 on: September 05, 2016, 09:53:47 PM »

the very high speed SD cards (and usb sticks) are SLC so can handle circa 100k wipes.  Should last a while.
Logged

sevenlayermuddle

  • Helpful
  • Addicted Kitizen
  • *
  • Posts: 5369
Re: Pi - SD card wear with mysql employment
« Reply #10 on: September 05, 2016, 10:28:23 PM »

To me, a bigger problem with flash memory is, how do you prepare it for disposal, ensuring that sensitive data has been removed?

Overwriting the data won't work, as the new writes might be mapped to different memory locations, thanks to wear-levelling algorithms.

With a traditional magnetic HDD, you can physically (with hammer and chisel) damage the platters.   NSA or GCHQ might still be able to recover some data but the average Russian hacker would not stand a chance.  Yet with flash, one would have to be sure to drive the chisel through each and every memory  chip?
Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: Pi - SD card wear with mysql employment
« Reply #11 on: September 05, 2016, 10:56:00 PM »

I imagine that a few seconds in a microwave oven would fry the chips (so to speak) but I guess it could be dangerous or possibly damage the oven?
Logged
  Eric

sevenlayermuddle

  • Helpful
  • Addicted Kitizen
  • *
  • Posts: 5369
Re: Pi - SD card wear with mysql employment
« Reply #12 on: September 05, 2016, 11:07:11 PM »

Who knows a low tech traditional deep fat fryer, at temperatures circa 200c, might be just as destructive as the modern microwave?

Probably wise to change the oil afterwards though, I'm not sure what my excuse would be in response to 'these chips taste funny'.   :D
Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: Pi - SD card wear with mysql employment
« Reply #13 on: September 05, 2016, 11:14:24 PM »

I'm not sure about that. Many silicon devices are rated at well over 200 C.
Logged
  Eric