2024-09-17
fixed in assp 2.8.1 *SPAM-Eliminator* build 24261:

fixed:

 - related to https://assp.thockar.com/forum/viewtopic.php?t=3700
   If enableINET6 was used, binding to a listener after an assp restart failed on some systems, because assp used the
   deprecated IO::Socket::INET option 'Reuse' also in calls to IO::Socket::IP, where this option is not supported.
   Now assp uses the 'ReuseAddr' option instead, which is supported in both modules.
   

changed:

- related to https://assp.thockar.com/forum/viewtopic.php?t=3690

  It is now possible to define a sub CorrectASSPcfg::extract_html_text.
  The HTML-parser in assp.pl will do a call to CorrectASSPcfg::extract_html_text if this sub exists in lib/CorrectASSPcfg.pm.
  This call is done in first place of HTML-parsing and depending on the return value (0/1), the HTML-processing is continued or stopped in assp.pl.
  In this sub the content, which is used for bayesian/hmm and regexes, can be manipulated in place ($_[0]).
  If a return value of '1' is provided by this sub, assp will not do any HTML-tag striping - you'll need to do all the stuff in this sub
  by your self (have a look in to sub extract_html_text of assp.pl)!
  Keep in mind to leave the HTML-tag structure intact, if you stripout any content - otherwise the HTML-parser my be unable extract text from
  the content, or will do unexpected things! For example: if you stripout a <span....> also remove the related end of the HTML-tag </span> -
  <p ...>...</p> , <div ...>...</div>

  for example:
  This code will remove HTML-content, which is not shown to endusers in their mail client - but would be used by assp for bayesian and
  HMM processing (possibly without any success - read the topic in the forum for more information).
  
  sub extract_html_text {
      $_[0] =~ s/<\s*span\s+style\s*=\s*"\s*display\s*:\s*none\s*;?\s*"\s*>[^<]*<\s*\/span\s*>//gois; # strip all hidden content
      return 0; # do not stop HTML-processing
  }  

added:

- related to https://assp.thockar.com/forum/viewtopic.php?t=3700
  If $disable_SO_REUSEPORT is set to 0 or 2 assp tries to use the SO_REUSEPORT socket option

our $disable_SO_REUSEPORT = 1;           # (0/1/2) disable the SO_REUSEPORT socket option - there is no assp version which ever used this option
                                         # notice: windows never has this socket option - so leave this value at 1
                                         #  0 - do not disable - try it, but if not supported by the OS it is not used and a load warning is produced for the module 'Socket'
                                         #  1 - disable this socket option and do not try to use it
                                         #  2 - do not disable - try it, but if not supported by the OS it is not used and silently ignored

  You may try to play around with this value, if you get unexpected errors for assp listeners while bind or reuse.


2024-09-01
fixed in assp 2.8.1 *SPAM-Eliminator* build 24245:

- after an upgrade of the perl module Net::DNS to version 1.46 RBL,RWL and URIBL were no longer working and caused slow processing
  of mails, because no expected DNS-Answer-ID was found

- if 'normalizeUnicode' was switched on and a unicode BLOCK or SKRIPT definition (for example: \p{Mathematical Alphanumeric Symbols} was
  used in a regular expression and a mail contained a related unicode character, which was normalized (e.g. to latin) - a match
  was not found by assp

- whoisip queries to RIPE caused some times persitent stucking workers on windows systems
  (other OS'es may be also affected by this issue - but were not reported at the time of writing)



2024-08-09
fixed in assp 2.8.1 *SPAM-Eliminator* build 24222:

- fixes typos

2024-08-09
fixed in assp 2.8.1 *SPAM-Eliminator* build 24222:


- perl 5.40.0 (5.040.000) is now supported 
- fixes several typos


2024-07-12
fixed in assp 2.8.1 *SPAM-Eliminator* build 24194:

- droplist downloads require the HTTPS:// protocol - the assp code and the version.txt contains now the new URL's




2024-06-17
fixed in assp 2.8.1 *SPAM-Eliminator* build 24169:

- If subaddressing with 'SepChar' was enabled, the subaddressing (change the email address) was only done for the envelope recipient.
  In case a subaddressed recipient address was also used in the MIME header 'to','cc:','bcc:' fields and the header address check 'DoHeaderAddrCheck' was enabled, this check failed,
  if the original email address was unknown.
  From now on, assp will do the subaddressing for the header address checks - but will not change the header addresses! If you need to change the header addresses as well,
  use the Recipient-Replacement feature instead of 'SepChar' - but keep in mind: this will break most of the MIME/DKIM/ARC ... signatures

- on windows systems, some times an unexpected 'error: unable to move ..../spam/... to ..../errors/notspam/... - No such file or directory' was logged in case of processing a resend request,
  even the move of the file was successful




2024-05-03
fixed in assp 2.8.1 *SPAM-Eliminator* build 24124:

- this release fixes several edge cases for blockreporting, where a file requested for a resend was not found by assp, even the file has exist in any of the collection folders

- if an admin resend request contained the extension - do not ....
  like in:
###the_requested_file_name.eml###do not scan

the - do not - was ignored, but: dont,don't,no,not - were working



2024-04-16
fixed in assp 2.8.1 *SPAM-Eliminator* build 24107:

- if 'maxSizeExternal' or 'maxSize' was set, assp added an additional '250-SIZE ddddd' line to the EHLO reply
  even the SIZE MIME-extension was already used, which may caused confusion at the receiving client site

changed:

- ASSP_AFC.pm 5.52 improves malware detection in javascript and java code



2024-03-21
fixed in assp 2.8.1 *SPAM-Eliminator* build 24081:

- ASSP_AFC.pm 5.51 fixes a problem - unexpected single [CR] (0x0D) were added before the mail-end sequence [CR][LF].[CR][LF]
  assp has corrected this mistakes, which may caused failed DKIM and SMIME signatures

- if DKIM signatures were added by assp and the mail contained incorrect line endings, the incorrect line endings were
  corrected after adding the DKIM signature - the DKIM signature may be failed for this reason
  if possible and required, those corrections are now done before the DKIM signature is added
  
   

2024-03-11
fixed in assp 2.8.1 *SPAM-Eliminator* build 24071:

- under rarely conditions it was possible that assp has not sent a proper SMTP-line-ending sequence [CR][LF], instead only a single [CR]
  or a single [LF] was sent - this is fixed

- under havy workload, it was possible that assp was going out of available SMTP-workers and unexpected restarted
  this was caused by a deadlock condition in the SPFRecCache cache
  
- slightly better overall performance, because of rebuilded internal regular expressions



2024-01-31
fixed in assp 2.8.1 *SPAM-Eliminator* build 24031:

- If the 'tc' header-flag was set in an UDP DNS-server-answer packet, the required TCP query was'nt done in every case - which caused unexpected DNS-query errors

- The maintenance of the internal SPF-record cache is improved. A refresh is now only done, if the TTL of any used DNS-record is expired.

- If the 'mx' tag was used in a SPF-record definition of a domain and a mx-record pointed to a hostname and the hostname resolved to multiple IP-addresses, only
  the first IP was used for the SPF-record cache.


added:

'DNSroundRobin' - Query DNS-servers in Round Robin mode
'If enabled, the available DNS-servers will be used by assp in round robin mode. Each DNS-query is sent to one DNS-Server at a time.
Which DNS-servers are used and also their order is random. Remaining DNS-servers are used as fallback.
This results in less DNS-queries - but increases the processing time, in case the used DNS-server does not answer fast.



2024-01-25
fixed in assp 2.8.1 *SPAM-Eliminator* build 24025:

- Large (several thousand entries/lines) IP-lists caused an CPU maxout for several seconds, if the compilation of the resulting regular expression has taken longer than one second,
  which is the default thread delay for config reloads in threads.
  Now the compilation in each thread is delayed until all lower threads have finished the configuration reload (regex compilation).

- a file change in any regex or IP config parameter caused a regex recompilation in all threads - even the resulting regex was not changed
  now the recompilation is skipped, if the resulting regex is not changed

- the time required to compile IP-address regular expressions is reduced

- the runtime (check) for IP-address regular expressions is reduced


changed:

- The perl module Schedule::Cron is no longer used by assp. The scheduler is now part of the assp.pl code. If a cron entry is used for a scheduled task,
  the configured entries are explained in human words (english only) in the GUI and possible syntax error are shown.
  Schedule entries can be analyzed/explained using the 'work with email addresses and domains' GUI dialog.



2024-01-09
fixed in assp 2.8.1 *SPAM-Eliminator* build 24009:

- ASSP_AFC 5.49 : embedded or attached .eml files may caused a processing loop (processing timeout)
- SNMP is now working again with strawberry perl 5.38.2 on windows x64 (download available at SF)
- lib/SNMPmakeMIB.pl got an update to correctly show worker status OID entries
- user based IP-address matching (entries like : a.b.c.d/x=>user@domain.org) was not working correctly in every case



2023-11-27
fixed in assp 2.8.1 *SPAM-Eliminator* build 23331:

- the memory footprint of the rebuild spamdb task is reduced
- the output for the size of the tmpDB folder in the rebuild report was wrong
- some GUI corrections
- wrong configuration of the database drivers DBD::ODBC and DBD::ADO crashed assp instead falling back to flat files
- used database drivers were missing in the output of notes/loaded_perl_modules.txt
- it was no longer possible to monitor the rebuild spamdb task in the worker status screen, because of too frequently updates by 'getEmailAddr'
- unknown characterset (eg: handcrafted and invalid) defintions in emails caused unexpected exception in several code parts of assp
- improved parsing and processing of (handcrafted) html code in text/plain email parts

changed:

- The text processing engine for bayesian and HMM is improved. It is recommended to run a rebuildSpamDB after upgrading to this version.



2023-09-08
fixed in assp 2.8.1 *SPAM-Eliminator* build 23251:

- In IP-address lists it is now possible to define an ASN (Autonomous System Number) for an IP-address.
  The IP-address-range for the IP in the ASN is resolved and used instead of the defined IP-address.
  Even the ASN contains more than this IP-address range, only the range in which the defined IP-address is included is used.

GUI explanation:
...
It is also possible to let assp lookup the ASN (Autonomous System Number) for an IP-address (NOT the ASN number its self - like ASN:1234). The CIDR of the ASN will be used by assp.
To lookup the ASN for an IP-address, write ASN:x.x.x.x or ASN:aaaa:bb::c
The ASN:ip-address notation can be also used for IP lists in a group definition.
...


added:

- ASSP_AFC.pm version 5.48 is now able to detect 'MHT MalDoc' (JPCERT/CC - https://blogs.jpcert.or.jp/en/2023/08/maldocinpdf.html) attacks.
  JPCERT/CC currently describs only the (one) case of handcrafted PDF files with MHT content.
  Simple tests have shown, that it is possible to include MHT's in many file types (for example images as well) and
  that MS-Office on Windows will open the MHT code, if the file extenson matches an MS-Office file extension - even the magic number of the file is not related to any MS-Office file.
  ASSP_AFC will detect MHT content in any attachment where such content is unexpected.
  Until now such files were only detected because of a possible missmatch between the file-magic-number (MIME-Type) and the file extension. 



2023-08-13
fixed in assp 2.8.1 *SPAM-Eliminator* build 23225:

- because of a DNS-answer layout change of asn.routeviews.org, the ASN (Autonomous System Number) for the IP-address was no longer shown in the analyzer output
  and in the results of the "work with IP-addresses" dialog




2023-08-02
fixed in assp 2.8.1 *SPAM-Eliminator* build 23214:


changed:

related to SF ticket(150) : DQS Service spamhaus.net is not working correctly

- The spamhaus offers a new service 'DQS Service spamhaus.net'. Since 06/2023 they are blocking legacy (spamhaus.org) requests from different ISP's.
  As an alternative they are offering a free and a payed DQS Service. Both require registration at spamhaus and an accesskey has to be provided in each query.
  The accesskey has to be provided the following way : query-data.your-accesskey.spamhaus.url
  As a result, assp has treated the key as part of the provider host, and the key was included in the log as well as in reply codes for URIBL and DNSBL/RBL.
  To prevent this, the accesskey has to be surrounded by curly brakets in the RBL/URIBL-Service-Provider defintion
  like : $DATA$.{your-accesskey-here}.zen.dq.spamhaus.net - if the accesskey needs to be placed anywhere in the middle.
  Older definitions, which starts with the accesskey, like youraccesskey-here.$DATA$.provider.org will still work



2023-07-17
fixed in assp 2.8.1 *SPAM-Eliminator* build 23198:


changed:

- an alias for the charset 'ISO-8859-8-I' is added to Encode, if the charset is not supported by the currently installed version of Encode

- the modus used to delay connections, if the connected IP matches the provided SPF-record, is changed
  until now, a hash was calculated over all IP-ranges in a SPF-record of a domain and the hash was used in tuplets and triplets
  this caused problems (delay loops), if other assp instances resolved the SPF-record from different DNS-servers or the SPF-record contained dynamic variables (the resulting hash has changed for any reason)

  from now on, the 'SPF:' tagged domain name is used for tuplets and triplets, if a SPF matching IP-address is found
  this way delaying is solved for a connection ( no matter if the SPF-record is anyhow different ) as long as the connected IP-address matches the SPF-record of the domain
  


2023-07-14
fixed in assp 2.8.1 *SPAM-Eliminator* build 23195:

- if a user,sender based (a privat) whitelist entry like user@local.domain,sender@other.domain was created using the 'work with lists' GUI dialog,
  an existing related personal-black-list entry was not removed

- if a mail contained an unknown charset definition (unknown to the perl module 'Encode') and this mail was stored in any corpus folder, the rebuildspamdb process
  died on an UTF8 exception - such mails/files may caused the same or similar exception in other assp operations
  recommendation: keep this perl module uptodate
                  NOTICE: Encode is a perl core module. If you use a distribution based perl, the module will be normaly updated with the installation of a new perl version.
                          Even there is a much more better version of this module available, most OS distributions are installing older versions of this module, if
                          not the latest perl version is installed. It is safe to update this module to the latest available on any perl version - using cpan or cpanm.

  example: installed Encode version 3.17 (perl 5.36.0) and a mail contains the MIME-charset defintion ISO-8859-8-I (https://en.wikipedia.org/wiki/ISO-8859-8-I)
           ISO-8859-8 is known to Encode, but it is not an alias for ISO-8859-8-I - so ISO-8859-8-I is unkown to this Encode version and this will cause the exception in assp
           Encode version 3.19 is aware of the MIME-charset ISO-8859-8-I by making ISO-8859-8 and alias for ISO-8859-8-I - but Encode version 3.19 is only distributed with perl 5.38
           Even you use the latest perl version, there may be updates for the module Encode available!
           


2023-07-06
fixed in assp 2.8.1 *SPAM-Eliminator* build 23187:


- added support for Perl 5.38.0


2023-07-03
fixed in assp 2.8.1 *SPAM-Eliminator* build 23184:

- if a backend-server (e.g. bad configured exim) offered the SMTP-extension PIPECONNECT and a client/server used the pipelining option, the connection failed after the DATA command

changed:

- the at least recommended version of the module Net::SSLeay is changed from 1.72 to 1.85

- the export extrem IP-List is now correctly sorted by IP

- it is now possible to find a match for an empty envelope recipient in 'bombSenderRe'


added:

- it is now possible to set the OpenSSL security-level (default is 1) - using any of the following methodes
  - in lib/CorrectASSPcfg.pm:   $main::openssl_security_level = 1;        # ( /1/2/3/4/5) used openssl security level - empty uses the libssl buildin value (default = 1)
  - SSLAdvancedServerConfigFile : SSL_CTX_set_security_level = 3;
  - SSLWEBConfigure: $parms->{'SSL_CTX_set_security_level'} = 3;
  - SSLSTATConfigure: $parms->{'SSL_CTX_set_security_level'} = 3;
  - SSLSMTPConfigure: $parms->{'SSL_CTX_set_security_level'} = 3;

- it is now possible to change the MIME-encoding of a mail before a DKIM-signature is added to it, if a specific MIME-header is found or the mail meets specific conditions
  to apply this behavior, define a sub DKIMconvCTE in lib/CorrectASSPcfg.pm
  if this sub is found by assp, it will be called by assp, providing the connection handle ($fh) and an array reference which can be modified inplace
  example:
  
sub DKIMconvCTE {
    my ($fh, $convCTE) = @_;
    my $this = $main::Con{$fh};
    
    if (! $this->{mailfrom}) {

        #                      ct(1)     ,   Regex for ct (2)    , target encoding (3)
        push @{$convCTE}, ['Content-Type','(?:text\/(?:ht|x)ml)','base64'];
#       push @{$convCTE}, ['.....' ,      '.....',               '....'];
#       push ...
#       push ..
#       ...
    }
}  

   in this example, if there is no envelope sender, assp will check the 'Content-Type' (1) of the mail against the given regular expression (2) - and if
   a match is found, the MIME-encoding will be changed to 'base64' (3) before the DKIM-signature is added
   the reason for this implementation is: some NDR's, delivery notifications, report mails, OoO-mails (e.g. mails with clear text attached rfc822 mails or mail headers) may cause
   the DKIM-signature to become invalid after passing the final MTA (if the MTA reformated the mail for any reason)


2023-05-16
fixed in assp 2.8.1 *SPAM-Eliminator* build 23136:

- if openssl 3.0.0 or higher was installed, the installed openssl library version was not shown in the "Perl Modules" status page


2023-05-11
fixed in assp 2.8.1 *SPAM-Eliminator* build 23131:


- version released for public - no code changes 
2023-05-10
fixed in assp 2.8.1 *SPAM-Eliminator* build 23130:

- the latest Net::DNS module version 1.38 throws a deprecation warning about the usage of the function rr->rdatastr (instead of using rr->rdstring)
  it is expected, that in any of the next versions of Net::DNS the function 'rdatastr' will be removed from Net::DNS::RR, in this case ALL older assp builds will no longer work
  


2023-04-26
fixed in assp 2.8.1 *SPAM-Eliminator* build 23116:

- the download URL's for the two- and third-level-tld files are changed to (the download was no longer woring using the old URL's)
https://www.surbl.org/static/two-level-tlds
https://www.surbl.org/static/three-level-tlds

- wrong crafted or looped SPF-Records may caused crashes while maintaining the SPF-Record-Cache (in tmpDB/files)

- Talos stopped redirecting http://senderbase.org - all related web-links in the GUI are changed

- the output of the analyzer for a wildcard (.*) good...-attachment-rule was wrong





2023-03-30
fixed in assp 2.8.1 *SPAM-Eliminator* build 23089:

- if the bulkimport of the spamdb and hmmdb were disabled for MSSQL-Databases, the import failed because of the usage
  of wrong table names
  
- it was possible that preHeaderRe matched in the analyzer but not in the real mail processing, because of header line splitting

 

2023-03-13
fixed in assp 2.8.1 *SPAM-Eliminator* build 23072:

******************************************************************************************************************************

*****************************
* !!!!!!! ATTENTION !!!!!!! *
*****************************
*    this version of assp   *
*      requires at least    *
*                           *
*        perl 5.12.0        *
*                           *
* it will NOT start on lower*
*      versions of perl     *
*****************************
* !!!!!!! ATTENTION !!!!!!! *
*****************************

Because several required perl modules changed their minimum required perl version to 5.12, the minimum required perl version
to run assp is also changed to 5.12.0

******************************************************************************************************************************

- ASSP_AFC.pm is upgraded to version 5.46
  if symbols were used in a infected or not allowed attachment name, the removal of the attachment failed

- 'RemoteSupport' was not working if 'webAdminPort' was configured without a defined IP-address (like: SSL:55555|55555)

- several regular expression optimization were incorrect, because the perl module Regexp::Assemble used a fast but lazy
  algorythm per default - assp now forces Regexp::Assemble to use the right algorythm


- if code execution was configured in a weighted regular expression on perl 5.34 or 5.36, assp rejected the execution of
  the regex while searching for the configured weight value


- if a group name contained upper case characters and was use in 'userAttach' , assp was unable resolve the group members,
  because it was searching for the (all) lower case group name



changed:

- the version changed from 2.6.x to 2.8.x and the code name is changed from *SPAM-Evaporator* to *SPAM-Eliminator*

- 'SepChar' now allows to define more than one character to support switching from one to another character, were it can
  be required to support both, the old and the new character, for some time


- 'removeDispositionNotification' can now also be used to remove any unwanted MIME-header by its tag

- 'URIBLCCTLDSRE' (the largest regular expression in the assp distribution) is now compiled in a separate process
  this prevents a stucking MainThread on some systems (under havy workload), while the regex is compiled in the MainThread

