MIME handling tests

Hi all,

As I’ve been doing a huge amount of work on the email processing feature of m2f 2.0, I’d like some assistance with testing.

The new functionality is in the Email Import channel. In short, it will convert a multipart email (i.e. one in plain text as well as HTML, or an email with attachments) into a single, flat message suitable for using later in an m2f chain.

For the present, all attachments are stripped from the incoming message, and saved into a directory on your filesystem. There’s a new File Manager class which servers up the files, given a unique URL. For example, on my test server, an attachment will be given a URL like this:

http://m2f/download?file=5863b365ee66b7a28546b4f07e09913b.

If that URL is entered into a browser window, m2f will offer the file for download.

I’d like you to test the following functionality:

• check that all processed emails are legible, and well-formatted
• check that all attachments are correctly stripped, and are downloadable from the specified URL

Testing the MIME handling

There is a specific sequence of operations you must follow to properly test the new functionality. Please follow the instructions carefully. You will need an archive of old emails to test with, in mbox format. Please try to test at least some with HTML parts, and attachments, where possible. This is the main focus of the test!

1. Work with a copy of your mbox file, as we will be deleting messages individually from it as the test progresses.
2. First, it’s imperative that you do a new checkout from SVN (the MIME parsing feature isn’t yet available in a release), and run the entire test suite. Any errors must be reported and resolved before there’s any point in proceeding.
3. Set up the following chain using the m2f Admin Control Panel:
   • Import channel: M2f_Module_Channel_Email_Import_Mbox
     • set filePath to the path to your mbox file
     • set maxMsgs to 1
     • set deleteMsgs to false (i.e. do not check the box)
   • Export channel: M2f_Module_Channel_FileWriter
     • set the filePath to the path of a file which is accessible using your webserver, with the suffix “.html”. For my tests, I use a file which can be accessed using the URL http://localhost/mime_test.html.
4. Run the chain. This should result in the contents of the first email in the mbox being parsed by m2f, and written to the file specified in the Filewriter export.
5. View the HTML file in your browser. If the original email contained any HTML, the generated file should be in HTML as well, and should already look good in your browser. If the original was plain text only, the contents of the HTML file will actually be plain text, so you’ll need to View Source in your browser to see the correct output. Please ignore the fact that a plain text message looks crap when viewed as HTML - that’s expected behaviour!
6. Check that the message looks good - it should be legible, with no missing images or text. Please note that it will not necessarily look the same as in your email client, as at present I’m stripping all CSS from the HTML headers. This may change in future.
7. Check that any attachments were correctly scrubbed, and that they are now accessible using the given URL. A block of text like this:
   An attachment was scrubbed:
Name: my_pic.jpg
Type: image/jpeg
URL : http://m2f/download?file=98070067013836d34358de6c092ef3c0should have been added to the bottom of the message for each attachment. Copy the URL, paste it into your browser, and you should get the option to download the file. Check the file is of the correct type (i.e. Microsoft Word documents open OK in Word, etc.), and that the contents is as expected.
8. If anything went wrong with the parsing - if the email looks weird, images are missing, attachment are not accessible, etc. - please do the following:
   • copy the RAW source of the email in question from your mbox file. It will be the  text starting from line 1, and going up to the next instance of this string: a new line, followed by the word “From”, followed by a space.
   • paste the raw email into a new text file. Please use the same encoding as the mbox file - if in doubt, use UTF-8.
   • replace any sensitive data with dummy data - but please don’t in any way alter the syntax of the email
   • email the file as an attachment to me at george@mail2forum.com, explaining what the problems were.
9. If the output looks OK, delete the raw email from the mbox (see above for how to identify a single email). This should leave the first line of the mbox file as the “From …” line of email number 2.
10. Run the chain again for the second email.

I’m very grateful for your help with testing this complex bit of code!