Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: IMAP downloader  (Read 3999 times)

sevenlayermuddle

  • Helpful
  • Addicted Kitizen
  • *
  • Posts: 5369
IMAP downloader
« on: November 11, 2013, 05:19:46 AM »

I wish to run an overnight script that downloads all of my gmail (several different gmail mailboxes) for backup purposes, leaving the originals on the server.   The same email needs to be accessible from a number of different clients (iPads, thunderbird on a different PC) etc, so I have to use IMAP.

So far I have been using 'getmail' which works pretty well, but can be a bit temperamental.   It recently threw a bit of a wobbly after being updated for example, insisting upon an unecessary (one-off) download of all mailboxes in entirety even though it is configured to only download new messages.   It has now resumed normal operation (new messages only) and is working fine, but my trust has been dented.

http://pyropus.ca/software/getmail

Comments elsewhere have suggested that our resident experts do not regard my setup as a robust combination.   Are there any easy to use alternatives to getmail?  Anybody else done this?

My 'getmail' wobbly occured on rpm (yum) managed fedora.   Is there any reason to think it wouldn't have happened when I updated getmail on any other distro', or if it had been installed other than by yum?
Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43552
  • Penguins CAN fly
    • DSLstats
Re: IMAP downloader
« Reply #1 on: November 11, 2013, 07:27:02 AM »

I've no experience of mail servers, but I get the impression that Postfix is quite popular. It's described as being very similar to Sendmail, but simpler internally and easier to set up.
Logged
  Eric

sevenlayermuddle

  • Helpful
  • Addicted Kitizen
  • *
  • Posts: 5369
Re: IMAP downloader
« Reply #2 on: November 11, 2013, 08:44:18 AM »

I've no experience of mail servers, but I get the impression that Postfix is quite popular. It's described as being very similar to Sendmail, but simpler internally and easier to set up.

Thanks Eric,

It's not a mail server I want, it's simply a retriever.   For sending mail from within scripts I find the basic 'mail' command does what I need.

I want to retreive mail from the remote server (gmail in this case) packaged up into files that can be read by other mail clients, it's just a back up to cover the unlikely scenario where google might let me down by losing all my mail.  It also came in useful on one occasion when I managed to accidentally delete a mail item that I regretted;  it was long gone on the server by the time I reallised, but I was able to retrieve it by digging out the corresponding backup.

getmail works a treat one you get it going, it's been running well for a year or so.   But an IMAP retrieval has the potential to do massive damage if it were buggy by deleting stuff on the server, so you need to have something you can trust.   :o

edit,  PS:  To clarify, Burakkucat has pointed out that I might have a lot more trust if I used a distribution other than Fedora.   That has been noted I'm already convinced he is right.   But come the day I build a new (not Fedora) server, I shall still face the dillema of whether to continue with getmail, or something else...   ???

« Last Edit: November 11, 2013, 09:18:27 AM by sevenlayermuddle »
Logged

tickmike

  • Kitizen
  • ****
  • Posts: 3640
  • Yes Another Penguin !. :)
Re: IMAP downloader
« Reply #3 on: November 11, 2013, 02:45:48 PM »

Hi.
I had a look in my PCLinuxOS package manager and there is program called 'IMAP utilities' and also 'Getmail' and 'fetchmail' .

About my backup server.
It's built around an old P IV and uses a very cut down PCLinuxOS 'Minime' Kernal (which never gets updated ! ) and that is mounted on a 8Gig SD Card (split in to 6 Gig for / and the rest for /Home ) .
It has two large 2 x 2TB  storage discs with 'Raid' software.
It's been running for two years (with no problems) since my daughters has been at uni and her laptop sends and automatic backup every day to my home server via SSH over the internet using 'LuckyBackup'.
Logged
I have a set of 6 fixed IP's From  Eclipse  isp.BT ADSL2(G992.3) line>HG612 as a Modem, Bridge, WAN Not Bound to LAN1 or 2 + Also have FTTP (G.984) No One isp Fixed IP >Dual WAN pfSense (Hardware Firewall and routing).> Two WAN's, Ethernet LAN, DMZ LAN, Zyxel GS1100-24 Switch.

sevenlayermuddle

  • Helpful
  • Addicted Kitizen
  • *
  • Posts: 5369
Re: IMAP downloader
« Reply #4 on: November 11, 2013, 04:42:03 PM »

Hi.
I had a look in my PCLinuxOS package manager and there is program called 'IMAP utilities' and also 'Getmail' and 'fetchmail' .

About my backup server.
It's built around an old P IV and uses a very cut down PCLinuxOS 'Minime' Kernal (which never gets updated ! ) and that is mounted on a 8Gig SD Card (split in to 6 Gig for / and the rest for /Home ) .
It has two large 2 x 2TB  storage discs with 'Raid' software.
It's been running for two years (with no problems) since my daughters has been at uni and her laptop sends and automatic backup every day to my home server via SSH over the internet using 'LuckyBackup'.

That's the kind of backup I like, physically 'off-site'.   :)

re getmail/fetchmail, I remember looking at both.  But one of the things I've forgotten is why I chose the former over the latter. Anything longer ago than last Tuesday, and I seem to struggle these days   :D

I think it was getmail's ability to write to the retrieved mail in 'mboxrd' format, which is compatible with Mozilla Thunderbird mail client, which I run on the Mac & the PC.  So if google were to suddenly evaporate, I can just copy all the downloaded mbox files into my thunderbird's directories and voilà…  there before my eyes will be all of my sent & received mail, right back to the 1990s.  So goes the theory, and pretty sure it worked to perfection when I tried it.  :)
Logged

tickmike

  • Kitizen
  • ****
  • Posts: 3640
  • Yes Another Penguin !. :)
Logged
I have a set of 6 fixed IP's From  Eclipse  isp.BT ADSL2(G992.3) line>HG612 as a Modem, Bridge, WAN Not Bound to LAN1 or 2 + Also have FTTP (G.984) No One isp Fixed IP >Dual WAN pfSense (Hardware Firewall and routing).> Two WAN's, Ethernet LAN, DMZ LAN, Zyxel GS1100-24 Switch.

sevenlayermuddle

  • Helpful
  • Addicted Kitizen
  • *
  • Posts: 5369
Re: IMAP downloader
« Reply #6 on: November 11, 2013, 05:51:09 PM »

Any good ?.
http://www.gentoo-wiki.info/HOWTO_Fetch_mails_from_multiple_POP/IMAP-mailboxes_and_export_them_via_IMAP_to_Thunderbird/SquirrelMail_etc

https://wiki.archlinux.org/index.php/Backup_Gmail_with_getmail

These links both describe pretty much what I have done, only my getmail scripts are run as part of a grander 'housekeeping' script of my own writing, that runs every night, backing up other devices on the LAN and taking snapshots of certain config files.

The trouble is, my confidence has been dented by the fact that getmail got itself out of sync after a system-wide software update.   I don't update the fedora system very often as it is rarely (if ever) used for web access or reading of email etc, so I reason that the security risks of being out of date are very low.   And if it ain't broke don't fix it!   

But the risks of running old software on any system connected to the internet are never zero, so once in a while conscience kicks in and I fire off an full update cycle, I need to be able to do that without losing continuity of getmail retrievals.

Hasten to add, in fairness to getmail, I still don't know if it was the getmail update that did the damage, or a side-effect of some other update.  :-\
Logged