diff --git a/app/models/channel/email_parser.rb b/app/models/channel/email_parser.rb index 64381d3f0..30452fa60 100644 --- a/app/models/channel/email_parser.rb +++ b/app/models/channel/email_parser.rb @@ -502,6 +502,12 @@ class Channel::EmailParser # strip all other tags string.gsub!( /\<.+?\>/, '' ) + # strip all & < > " + string.gsub!( '&', '&' ) + string.gsub!( '<', '<' ) + string.gsub!( '>', '>' ) + string.gsub!( '"', '"' ) + # encode html entities like "–" string.gsub!( /(&\#(\d+);?)/x ) { |item| $2.chr @@ -514,13 +520,23 @@ class Channel::EmailParser if hex chr = hex.chr if chr - chr + chr_orig = chr else chr_orig end else chr_orig end + + # check valid encoding + begin + if !chr_orig.encode('UTF-8').valid_encoding? + chr_orig = '?' + end + rescue + chr_orig = '?' + end + chr_orig } # remove empty lines diff --git a/test/fixtures/mail20.box b/test/fixtures/mail20.box new file mode 100644 index 000000000..e23384c4d --- /dev/null +++ b/test/fixtures/mail20.box @@ -0,0 +1,965 @@ +From drugs-cheapest8@sicor.com Mon Feb 10 03:59:59 2014 +Return-Path: +X-Original-To: info2@znuny.com +Received: from smtp1.sicor.com (unknown [95.57.99.218]) + by arber.znuny.com (Postfix) with ESMTP id 15AC5602C8 + for ; Mon, 10 Feb 2014 03:59:57 +0100 (CET) +From: Health and Care-Mall +To: +Subject: The Highest Grade Drugs And EXTRA LOW Price . +Date: Mon, 10 Feb 2014 08:59:52 +0600 +MIME-Version: 1.0 +Content-Type: text/html; charset="windows-1252" +Content-Transfer-Encoding: quoted-printable +X-UID: 505 +Status: RO +Content-Length: 56321 +Lines: 946 + +_________________________________________________________= +_______________Yeah but even when they. Beth li=EE ed her neck as = +well +
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
óû5aHw5³½IΨµÁxG⌊o8KHCmς9-Ö½23QgñV6UAD¿ùAX←t¨Lf7⊕®Ir²r½TLA5pYJhjV gPnãM36V®E89RUDΤÅ©ÈI9æsàCΘYEϒAfg∗bT¡1∫rIoiš¦O5oUIN±IsæSعPp Ÿÿq1FΧ⇑eGOz⌈F³R98y§ 74”lTr8r§HÐæuØEÛPËq VmkfB∫SKNElst4S∃Á8üTðG°í lY9åPu×8>RÒ¬⊕ΜIÙzÙCC4³ÌQEΡºSè!XgŒs.
çγ⇓BcwspC L I C K  H = +E R Eëe3¸ !Calm dylan for = +school today.
Closing the nursery with you down. Here and made the = +mess. Maybe the o=EF from under his mother. Song of course beth touched = +his pants.
When someone who gave up from here. Feel of god knows = +what.
TBϖ∃M5T5ΕEf2û–N¶ÁvΖ'®⇓∝5SÐçË5 Χ0jΔHbAgþE—2i6A2lD⇑LGjÓnTOy»¦Hëτ9’:Their mother and tugged it seemed like
d3RsV¶HÓΘi¯B∂gax1bîgdHä3rýJÿ1aIKDz n1jfaTk³Vs395ß C˜lBl‘mxGo0√úXwT8Ya õ8ksa∫f·ℵs”6ÑQ ÍAd7$p32d1e∏æe.0”×61aîΚ63αSMûNf5ÉCdL∪1i↔xcaa5êR3l6Lc3iãz16só9èU zDE²aEȨgs25ËÞ hE§cl⊃¢¢ÂoÒµBw²zF© qÏkõaXUius1r0⊆ d•∈ø$¢Z2F12­8l.07d56PÚl25JAO6
45loVóiv1i2ãΥ⌊að⊃d2gÃΥ3™rÎÍu¸aWjO8 n40–Soyè2u¡∅Î3p¢JΜNeÌé×jráÒrΚ 1ÌÓ9AúrAkc8nuEtl22ai‡OB8vSbéσeιõq1+65cw Òs8Uaò4PrsE1y8 ⟨fMElhϒ⋅Jo8pmzwjˆN¥ wv39aW¡WtsvuU3 1aœ³$éΝnR2OÏ⌉B.∀þc→5Ê9χw5pÃ⁄NfHGFVfE³ãiσjGpa5¶kgg¡ìcWrUq5æakx2h 0Fè4P¸ÕLñrn22ÏoþÝÐHfoRb2eUαw6sñN‾ws¶§3ΒiòX¶¸ofgtHnR⊥3âase9álF¿H5 à6BÁa⊃2iϒsô¡ói ÅkMylÚJ¾ÄoQ–0ℑwvmùþ = +ˈμ"aQ7jVse6Ðf «hÜp$Lâr£3i1tÚ.323h5qP8g0♥÷R÷
·iƒPV1Β∋øiF¤RÃa4v3âgL9¢wr¨7ø×aÏû0η þ1àßStuÞ³u7á¡lpÑocEe·SLlrVàXj ⊥Uµ¢F¬48ðov7¨Arm×4ÍcùVwÞe1§⊇N ÂÛ4äaLþZ2ski×5 c€pBlûù6∂olÃfÚwKß3Ñ 4iíla4C³êsREÕ1 ãeIó$âz8t442fG.¸1≤¸2F’Ã152in⊄Tl©ëC2v7Ci7·X8a×ú5NlþU⟩ιicO∑«s·iKN UuϒjSÃj5Ýu÷Jü§pn5°§e¥Û3℘rÆW‡ò J‹S7A1j0sc&ºpkt·qqøiZ56½vn8¨∗eîØQ3+7Î3Š ∑RkLaKXËasÐsÌ2 = +ïÇ­¶lDäz8oã78wwU–ÀC T6Uûaϒ938sÌ0Gÿ Oxó∈$98‘R2ÂHï5.ÒL6b9θrδÜ92f9j
Please matt on his neck. Okay matt hu=EF ed into your mindSince her head to check dylan. Where dylan matt got up there
1ȱΑAYQªdN¬ÚϒXT00ÀvI∨ío8-½b®8AΕºV4LgÕ↑7LKtgcEiw­yR5YýæGRA1°I¿0CïCàTiü/þwc0Ax211SÜÂùŒTÁ2êòHpNâùM6Ⱦ0A5Tb»:Simmons and now you = +really is what. Matt picked up this moment later that.
25¯yV9ÙßYeg·↑DnJ3l4tÝæb1os∏jll÷iSÐiwBÎ4n0ú1Ö ªf÷Ña§1løsuÚ8ê 2LCblgvN½o¼oP3wn♠90 FZora&M™xsΚbb ç5Ãξ$Âô·×2iGæ∇1⊇Ξ¬3.0P0κ53VÁö03ÝYzøX¢BAZ4KwdduÜvvuB↑ΒaÄ’THi0—93rZεj0 §rΜÅa2­·§s7¸Ιf 8⇓þolW„6Ýo6yH¥wKZ∧6 21hÒaKJ“ℜs48IÌ ÔÀ¬­$ZΣ¹ü2ñÙ6B42YMZ.Ô¹V¼9f·0å54⌈R8
÷w"9N2gBÀaðSê¢s≅gGÔo0Dn4n↵γ7⊗eS7eýxf3Jd q÷CMaÍä³isNMZp zz0˜lΚLw8oë29ww¤§Qu = +¥D⌈íaýË¢ésJ8Á¬ 3oùÙ$¦1Nℜ1>Rét7WPM¨.¶8¹D92k5D9∗8≈Rl©3ªSj·Ψ8pΣïKùi6rrÔrbÛu¬i2V∗∏v5ª10a27BÁ = +Ú♦Ξsa9j3χsa¯iΟ Oi℘ml6óféowbz∀wA6ù→ ñ×bàai´wbs♦βGs Ù81i$iÀˆ12⊃2wC82n8o.µ3NJ9S1©Θ0P1Sd
What made no one in each time.Mommy was thinking of course beth. Everything you need the same = +thing
PïEVGÿ9srEx⇐9oN3U®yEÎi2OR5kÇÿAΤηνULP¿∧q R5¿FHt7J6E»¯C∅Aå∃aVLu∗¢tT⟨2ÚHq9Né:
⊥ÞÞ¨T¦ªBrrC7³2adš6lmzb¨6ai07tdBo×KopíΡÄlj4Hy ÝaÓ1aÖí∉Ós1aá’ 4D­kleowËo3–1ÍwjR≤Π £RhÈafà7≅sù6u2 8NLV$∪⇓»↓1Y¶2µ.vßÈ23ÖS7û0Ün¬Äm5VKZy3KÎiñë¹DtÚ2HrhGaMvr5ïR«oÂ1namΜwÐãanFu8x7⌈sU E4cva£Âε™s7ΑGO dA35ldñÌèoAξI1wXKïn f¼x¾a∏7ffs†ìÖð 5msC$7Ët¦0z„n÷.it¡T7O8vt5¼8å·
Jï1ÏPkáO¶rnùrAo8s5∅z—4Rha1®t˜cq5YΧ ΤQÍraÑ⌋4¹sÜ5²§ ûVBιluwóioL3ëBw£±1¶ 5∈àáa1IÊ2sšÛÛ G´7ρ$kJM80∼∠ℵl.J1Km32µÚ⊃5ã鼧p°ÿ­A¹NU0c¥xçfo⟨Øácm14QGpHEj7lnDPVieV2¶aΠ2H7 ²j26azBSesë1c9 ´2Ù¬l0nò¤oõâRVw¦X´Ï = +αVõ­a≅σ¼Zs§jJå 3pFN$¾Kf821YΟ7.3ÍY95JΑqŸ0v9ÄQ
ñ↑yjPΤ1u6rFwhNeCOϖúd5Γêcne¼a0iTF¹5sxUS0o88ℵªlaÅT℘oOBÀ¹në·­1e∧Kpf υ98ξabp†3sj8â& 9©BolÎAWSo7wNgwø¦mM tteQat0ϖ2s4≡NÇ ÕƦΘ$ùRÓq0·Ã7ª.mt¾³1—uwF57H♣fæ∪HYSjψ3Byš²g¤ndXÀ5tµ¯ò6hZ⇒yÿr8ÿmdowyðdiψ8YΗd0ršŠ N0Ý9aÃ3I¦sQaýê Õ0Y7lZ¯18o∫50Çwµ"©Ζ n6Ü≥a∇lßnsF›J9 ºDΟK$Á4ÉL0S7zÖ.Ta2X3²R995391¡
Turning to mess up with. Well that to give her faceAnother for what she found it then. Since the best to hear
GX°♦Ca2isA¾8¡bNÉî8ÂAöÜzΘD∇tNXIfWi–Ap2WYNYF®b ≠7yφDpj6©R04EÂU´ñn7GÆoÌjS³Á∋TC⊥πËO1∗÷©RtS2wE66è­ = +νÑêéASi21DP“8λV∧W⋅OAÖg6qNtNp1T269XA7¥À²GGI6SEwU2íS3Χ1â!Okay let matt climbed in front door. Well then = +dropped the best she kissed
¤ÊüC>ΦÉí© flQkWMŠtvoÐdV¯rT´ZtlN6R9dZ¾ïLwuD¢9i3B5FdcÆlÝeSwJd KªtDDfoX±evrýwlK7P÷i§e³3vÎzèCe¬Μ♣ΝrGhsáy°72Y!gZpá R6O4O»£ð∋r9ÊZÀdB6iÀeîσ∼ÓrCZ1s ²ú÷I3ÁeÒ¤+⌉CêU »k6wG´c‚¾o60AJoR7Ösd3i¿Ásððpt Øè77añ∀f5np¤nþduE8⇒ ȹSHGJVAtew∇LëtςëDæ 6kÌ8FgQQ⊂R8ÇL2EI2∉iEHÍÉ3 Hÿr5Af1qximςρ‡r6©2jmWv9ÛaWð¸giACÜ¢lM⌋¿k = +ÊVÚ¸SÓùθçhµ5BΙi∗ttEp8¢EPpSzWJi32UÎn5ìIhgx8n⌉!j∏e5
x¯qJ>mC7f 5ºñy1GA4Ý0lCQe09s9u%uksã ψìX5A4g3nu←Τyst7ÍpMhšgÀÖe⟩pÚ£n¼YƒŠtÉÚLGizqQ↓c3tÙI œïbXMKÛRSertj×d"OtÊss58®!oo2i FÂWáEWøDDx7hIÕpΦSôBiÒdrUr⇔J<Õa1Αzwt0°p×ià8RÌoHÛ1Än¥7ÿr = +¯¥õàDYvO7aká»htì04Πe∂λÇ1 1ÈdUoο°X3fc63¶ e&∪GOxT3CvXcO·e3KËνr3¸y2 26Ëz3Ã∞I± Pì∃zYt6F4e6è⇓va5÷þ9rkΘ3äsKP5R!ιµmz
3í1ë>ð2′L 2óB⊥S∩OQMeý∉ÑΦcöè9Tuãa∫drâ5ûMeLk9Ô £æ1OOø9oKnÿψÀWl7HÏ∅i9ρÈÊniâ•ÛeXPxí ´Í5¡SUqtBh7æa5otSZ9pØËÛDpf®ÝÊiÛωbjn¯½Ÿ2gsçh− båÌswxðoSiq8hvtèé6Òh⌈b²S ×6þSVBEFCiøUàds9ѤΕaƧξÜ,1„wv jw7AMKÈ↔laæG9¦së3«etuB2keDãæìr°¨IeC¾EaÄao÷″∧r>6e¸d9DùÇ,mtSö I∗44A¹RˆêM98zME≅QŸÐX¹4j6 î0n3a1'Êânxpl6d83þJ 06Ð9Eïãýã-28Ú9c4ßrØh7è¥med½♠kcñ3sPk¶2•r!⟩QCa
ŠeÏÀ>Ãσ½å bpøNERN8eaD6Åns7Abhy±Æü∩ D7sVR8'ºEeÿáDVfc˜3ëu7ÏÆqncË3qdÊ∼4∇sρmi5 6æ¾Êaä°∝TnQb9sdÀMùℑ ∑gMÿ2bNð¶4cä½⊆/4X1κ7¥f1z ϖ1úECzf•1uMbycs1•9¾ts0Tào3hêDmSs3Áe7BíÉrô⋅ãÔ φ8Ä″SSXð¤uúI¸5p58uHp2cß±o∂T©Rrd6sMt∪µµξ!é4Xb
+
+Both hands through the = +fear in front.
Wade to give it seemed like this. Yeah but one for any = +longer. Everything you going inside the kids. + diff --git a/test/unit/email_parser_test.rb b/test/unit/email_parser_test.rb index 836fef9a3..68b259b15 100644 --- a/test/unit/email_parser_test.rb +++ b/test/unit/email_parser_test.rb @@ -76,7 +76,7 @@ Liebe Grüße! }, { :data => IO.read('test/fixtures/mail6.box'), - :body_md5 => 'fb6654b0171261e0cc103e63af75407b', + :body_md5 => '88c0c9e004021a4ed2a0c1e5f6b3455d', :params => { :from => '"Hans BÄKOSchönland" ', :from_email => 'me@bogen.net', @@ -86,11 +86,11 @@ Liebe Grüße! Test1:8 -Test2:& +Test2:& Test3:∋ -Test4:& +Test4:& Test5:= @@ -102,7 +102,7 @@ Test5:= }, { :data => IO.read('test/fixtures/mail7.box'), - :body_md5 => '775a69acf8ba0495712a3953f2ecff6a', + :body_md5 => '6029e6b6106a6dd11ed887ec31f118ac', :params => { :from => 'Eike.Ehringer@example.com', :from_email => 'Eike.Ehringer@example.com', @@ -115,11 +115,11 @@ Lass uns evtl morgen Tel. Mfg eike Martin Edenhofer via Znuny Team --- Installation [Ticket#11392] --- -Von:"Martin Edenhofer via Znuny Team" <support@example.com>Aneike.xx@xx-corpxx.comDatum:Mi., 13.06.2012 14:30BetreffInstallation [Ticket#11392] +Von:\"Martin Edenhofer via Znuny Team\" Aneike.xx@xx-corpxx.comDatum:Mi., 13.06.2012 14:30BetreffInstallation [Ticket#11392] Hi Eike, anbei wie gestern telefonisch besprochen Informationen zur Vorbereitung. a) Installation von http://ftp.gwdg.de/pub/misc/zammad/RPMS/fedora/4/zammad-3.0.13-01.noarch.rpm (dieses RPM ist RHEL kompatible) und dessen Abhängigkeiten. -b) Installation von "mysqld" und "perl-DBD-MySQL". +b) Installation von \"mysqld\" und \"perl-DBD-MySQL\". Das wäre es zur Vorbereitung! Bei Fragen nur zu! -Martin @@ -360,6 +360,17 @@ Hof :to => '377861373 <377861373@qq.com>', }, }, + { + :data => IO.read('test/fixtures/mail20.box'), + :body_md5 => 'd2b65203aaf2bbbd50fc73cb14d781bc', + :params => { + :from => 'Health and Care-Mall ', + :from_email => 'drugs-cheapest8@sicor.com', + :from_display_name => 'Health and Care-Mall', + :subject => 'The Highest Grade Drugs And EXTRA LOW Price .', + :to => 'info2@znuny.com', + }, + }, ] files.each { |file| @@ -373,9 +384,9 @@ Hof # check params file[:params].each { |key, value| if key.to_s == 'body_md5' -# puts 'md5' -# puts '++' + data[:body].to_s + '++' -# puts '++' + file[:params][key.to_sym].to_s + '++' + puts 'md5' + puts '++' + data[:body].to_s + '++' + puts '++' + file[:params][key.to_sym].to_s + '++' assert_equal( Digest::MD5.hexdigest( file[:params][key.to_sym].to_s ), Digest::MD5.hexdigest( data[:body].to_s ) ) else assert_equal( file[:params][key.to_sym], data[key.to_sym], "check #{key}" )