jeudi 10 mars 2011

mod_rewrite: http to https, https to http except specific files

Let's quote Brian Moore :

"Despite the tons of examples and docs, mod_rewrite is voodoo. Damned cool voodoo, but still voodoo."

Well, it ain't that bad... As long as you know how to use regular expressions and understand how rules are processed by your HTTP server, you're in business.

But on with common requirements and their practical solutions...

So you got your free SSL certificate ;) and you want a complete directory_path to use https:// instead of http://.

This is a .htaccess file you could put in that directory_path:
RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteCond %{REQUEST_URI} directory_path
RewriteRule ^(.*)$ https://%{SERVER_NAME}/directory_path/$1 [R,L]
For example, if this was on the server foo.com and the directory_path was /secure, someone typing http://foo.com/secure would get redirected to https://foo.com/secure.

Now let's say you want all requests for https://foo.com to be rewritten as http://foo.com, except for two specific files, say secure1.php and secure2.php. Here's one way to do it with .htaccess:
RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteRule ^secure1\.php https://%{SERVER_NAME}/secure1\.php [R,L]
RewriteCond %{SERVER_PORT} 80
RewriteRule ^secure2\.php https://%{SERVER_NAME}/secure2\.php [R,L]

RewriteCond %{SERVER_PORT} 443
RewriteCond %{REQUEST_URI} !secure1\.php
RewriteCond %{REQUEST_URI} !secure2\.php

RewriteRule ^(.*)$ http://%{SERVER_NAME}/$1 [R,L]
Ok, this is not the most elegant rule-writing code... We could use rule negation and have a little less rules, but I find the code above explicit and easy to understand - and blog-friendly! :) Also, we explicitly exclude secure1.php and secure2.php from the https:// to http:// RewriteRule, that's because we're using .htaccess and we need to avoid loops, as .htaccess would be read again after the redirect. If we were using per-server context with httpd.conf, we probably wouldn't need this.

Let's finish with Brian Behlendorf's quote :

"The great thing about mod_rewrite is it gives you all the configurability and flexibility of Sendmail. The downside to mod_rewrite is that it gives you all the configurability and flexibility of Sendmail.''

mercredi 9 mars 2011

Get your free SSL certificate.

There are plenty of trials out there (like Comodo's 90-day trial), but if you want to SSLize your website (yup, that word exists in my dictionary) for nothing at all, go to StartSSL.

It is NOT like self-signing a certificate, which will give you that annoying message saying that your site cannot be trusted, but it will give you a legitimate certificate confirming that you have been email- or domain-verified. Which is just enough to get that SSL-encrypted data flow, with just enough validation to avoid tickling your clients' browsers.

Yes, StartSSL's website could definitely be pimped-up, and it could provide with a nice-looking Verisign-like logo, but heck, if you don't have $400US to dish out, then it's the way to go. The process isn't that painful, you may need to refresh your browser once in a while to get through key creation, but it does create the CSR for you and it also provides with (slightly outdated) instructions to configure your web server.

Now who's going to provide us with that silly EV green bar for free? :p

lundi 7 mars 2011

Solaris + ZFS + MySQL + InnoDB performance tweaks

So I've got a relatively cheap ZFS system running in my garage, acting as a MySQL database server for data mining... It's running Solaris 11 Express with MySQL, Apache and PHP on it.

ZFS is a great filesystem, but you can make it better with some performance tweaks (besides using mirrored SSD's for your ZIL and L2ARC).
  • Compress your database pool(s):
  • zfs set compression=lzjb data/mysql
    zfs set compression=lzjb logs/mysql
  • Set the ZFS recordsize to match the InnoDB block size (16KB for InnoDB data files, and 128KB for InnoDB log files) :
  • zfs set recordsize=16K data/mysql
    zfs set recordsize=128K logs/mysql 
  • Limit ARC cache to < 80 % total RAM (in my case, 10GB for 16GB total RAM)
  • vi /etc/system:
         set zfs:zfs_arc_max = 10737418240
Use the following my.cnf directives:
[mysqld]

# use one large data file on one ZFS pool, with large autoextend
# /var/lib/mysql is symlinked to /data/mysql, a ZFS pool
innodb_data_file_path = ibdata1:1G:autoextend
innodb_autoextend_increment = 128

# put logs in a separate ZFS pool
innodb_log_group_home_dir=/logs/mysql

# use one file per table
innodb_file_per_table=1

# not too high (default=unlimited) to avoid thread trashing
innodb_thread_concurrency=16

# use 2GB because we have 16GB and ZFS needs a lot of RAM. I would use 4GB but MySQL does not accept a total log size > 4GB, which is abnormal for 64-bit...
innodb_buffer_pool_size=2G

# 25% to 50% of the buffer pool size to avoid buffer pool flush on log file overwrite
innodb_log_file_size=1G

# log buffers are flushed every second, so there's no reason to have a big buffer
innodb_log_buffer_size=8M

# not very important, InnoDB will get more RAM from OS if needed but warns in log
innodb_additional_mem_pool_size=256M

# compromise performance for full ACID compliance
innodb_flush_log_at_trx_commit=1

# traditional auto-increment
innodb_autoinc_lock_mode=0

# disable DNS lookups
skip-name-resolve

dimanche 6 mars 2011

Configurer ProFTPd avec des utilisateurs virtuels

Il est fort recommendable de créer des utilisateurs FTP "virtuels", c'est-à-dire qui ne font pas partie du système d'authentification du système d'exploitation (SE), au cas où le mot de passe d'un compte FTP soit découvert (et nous savons qu'il est facile d'obtenir cette information puisqu'elle n'est pas encryptée lors de l'authentification de la session FTP).

Sous Linux, plusieurs serveurs FTP existent, et nous verrons ici comment configurer le serveur ProFTPd pour qu'il utilise une table d'authentification différente de celle du SE (plus particulièrement, un fichier texte). Personnellement, j'aime aussi beaucoup vsftpd puisqu'il n'a presque aucune dépendance, et il se peut fort bien que cela fasse l'objet d'un autre message sur mon blogue.

De plus, nous créérons un compte FTP "webmaster" qui n'aura le droit que de modifier le contenu d'un site web.

Alors voici la marche à suivre :
  • Installer ProFTPd, que ce soit avec votre packager (apt-get, yum, zipper, etc.) ou manuellement en téléchargeant la dernière version du site officiel.
  • Vérifier sous quel groupe ID votre serveur HTTP roule (apache2 créé et utilise souvent le groupe www-data); dans mon cas, le GID de www-data est 33
  • Modifier le fichier de configuration /etc/proftpd/proftpd.conf en s'assurant d'avoir les lignes suivantes :
DefaultRoot                   ~
AuthOrder                     mod_auth_file.c
AuthUserFile                  /etc/proftpd/ftpd.passwd
Umask                         006 007
  • L'option DefaultRoot et le paramètre ~ restreignent les utilisateurs à leurs répertoires racines personnels.
  • L'option AuthOrder et le paramètre mod_auth_file.c obligent ProFTPd à authentifier les utilisateurs à partir d'un fichier texte.
  • L'option AuthUserFile et le paramètre /etc/proftpd/ftpd.passwd indiquent l'emplacement du fichier d'authentification utilisé par ProFTPd.
  • L'option Umask et les paramètres 006 et 007 spécifient les autorisations permises sur les fichiers et les répertoires créés par le serveur FTP, soit -rw-rw---- (660) et drwxrwx--- (770) respectivement.
  • Il reste à créer le compte FTP virtuel webmaster :
sudo ftpasswd --passwd --name=webmaster --uid=33 --gid=33 --home=/var/www --shell=/bin/sh
  • Il est important d'utiliser le GID trouvé plus haut (celui qu'utilise votre serveur HTTP) afin de donner les droits au nouvel utilisateur virtuel d'écrire dans le répertoire du site web. J'ai choisi d'utiliser le UID de www-data également (facultatif), pour que tout fichier transféré via FTP sous le compte webmaster soit créé en tant que l'utilisateur www-data et le groupe www-data. Nous utilisons ce groupe afin de permettre aux scripts PHP (roulés par le serveur HTTP) de lire et d'écrire les fichiers créés par le serveur FTP, et vice-versa.
  • Il ne vous reste plus qu'à définir les autorisations du répertoire de votre site web (souvent /var/www). Par exemple, pour permettre à tout utilisateur du système d'exploitation (SE) appartenant au groupe www-data de lire et d'écrire dans ce répertoire, il suffit d'effectuer les commandes suivantes :
sudo chgrp -R www-data /var/www
sudo chmod -R g+w /var/www
  • Si vous voulez qu'un utilisateur du SE puisse modifier, ajouter, effacer, etc. des fichiers du répertoire du site web, il suffit donc de l'ajouter au groupe www-data de votre SE (dans /etc/group).
  • Pour ne pas divulguer l'identité des réels propriétaires du contenu du répertoire de votre site web (par exemple, les fichiers qui auraient été créés par un utilisateur de votre SE appartenant au groupe www-data, vous pouvez utiliser les directives suivantes dans proftpd.conf qui auront pour effet de lister tous les répertoires et fichiers comme appartenant à www-data :
DirFakeUser on www-data
DirFakeGroup on www-data
  • Finalement, afin de ne pas divulguer la version de votre serveur FTP, vous pouvez changer la bannière de bienvenue en utilisant la directive suivante :
ServerIdent on "FTP server."
Voici le contenu du fichier proftpd.conf, au complet, avec la configuration de base tel que décrite ci-dessus :
Include /etc/proftpd/modules.conf

UseIPv6                         off
IdentLookups                    off
UseReverseDNS                   off
ServerIdent                     on "FTP server."
DeferWelcome                    on
ServerType                      standalone
MultilineRFC2228                on
DefaultServer                   on
ShowSymlinks                    on
TimeoutNoTransfer               600
TimeoutStalled                  600
TimeoutIdle                     1200
DisplayLogin                    welcome.msg
DisplayChdir                    .message true
ListOptions                     "-l"
DenyFilter                      \*.*/
DefaultRoot                     ~
RequireValidShell               on
Port                            21
MaxInstances                    30
User                            proftpd
Group                           nogroup
DirFakeUser                     on www-data 
DirFakeGroup                    on www-data 
Umask                           006 007
AllowOverwrite                  on
AuthOrder                       mod_auth_file.c
AuthUserFile                    /etc/proftpd/ftpd.passwd
TransferLog                     /var/log/proftpd/xferlog
SystemLog                       /var/log/proftpd/proftpd.log


QuotaEngine off



Ratios off



DelayEngine on

samedi 5 mars 2011

VMware tools for 32-bit Debian 6 (Squeeze) guest on 32-bit VMware host (Server 2 or ESX(i) 3.5)

Debian 6 (Squeeze) is now the distro's official release. You want it, even if the installation disc's welcome screen totally sucks. Come on Debian guys, you can do better than that. Anyways.

Installing the latest Debian as a 32-bit VMware guest is pretty straightforward, except when it comes to installing the VMware tools, especially on 32-bit VMware hosts. You will most probably get errors if you try to compile the tools included with VMware Server or if your ESX(i) 3.5 host is not up-to-date (and we all know it's not always easy to update an ESX(i) 3.5 host via VMware Infrastructure Client)...

So, whether you're using ESX(i) 3.5 or VMware Server 2 as a host, I found the easiest way to install the VMware tools for your latest and greatest 32-bit Debian guest is to do get the latest version from VMware and to install them manually. Here it goes:
  • download latest patch by selecting appropriate VMware host
  • from the downloaded .zip file, find and extract the file VMware-tools.tar.gz (you may need to traverse other .zip files within this one)
  • extract the linux.iso file from that archive
  • mount the ISO on your debian guest, using either the host's GUI or by transferring and mounting it directly from within your guest
  • find the .tar.gz file that contains the tools. As of today, the latest one for VMware 32-bit is VMwareTools-3.5.0-317866.tar.gz.
  • proceed with the usual installation procedure of the VMware tools

    vendredi 4 mars 2011

    Install Guide for Request Tracker 3.8.4 on SLES 11

    Install Guide for RT 3.8.4 on SLES 11
    (also available on the RT Wiki)

    Scope:

    This install guide is for RT 3.8.4 on a minimal, no-GUI SLES 11 machine used only for RT. This document takes care of OS installation and configuration of RT, and does not touch basic email setup (such as postfix or sendmail configuration).

    Additional Notes:

    It is recommended to get an activation code for SLES on the Novell website to do the online updates. SLES 11 may be installed on baremetal or as a virtual guest. In this case, I've installed the 64-bit, multiprocessor version as a ESXi 4 guest, in full virtualization mode. Memory usage is approx. 256Mb in idle mode, and disk usage is approx 3Gb.

    Versions:
    cat /etc/SuSE-release:
         SUSE Linux Enterprise Server 11 (x86_64)
         VERSION = 11
         PATCHLEVEL = 0
    
    uname -srio:
         Linux 2.6.27.23-0.1-default x86_64 GNU/Linux
    
    httpd2 -v:
         Server version: Apache/2.2.10 (Linux/SUSE)
         Server built:   Feb 23 2009 21:08:14
    
    SLES 11 Installation Settings:
    • Remove Gnome, X Windows, Print Server, Web-based Enterprise Management
    • Add C/C++ Compiler and Tools
    • Disable IPv6
    • Disable Firewall
    • If DHCP-configured with access to Internet: Enter Activation Code and run Online Update (you may need to do this many times)
    • Do not create CA and Certificate (unless you use LDAP)
    Static IP Configuration:

    Configure IP address:
    vi /etc/sysconfig/network/ifcfg-eth0:
         BOOTPROTO =’static’
         IPADDR=''yourIP"
         NETMASK="yournetmask"
    
    Add default route:
    vi /etc/sysconfig/network/ifroute-eth0:
         default yourgatewayIP 0.0.0.0 eth0
    
    Add name resolver :
    vi /etc/sysconfig/network/config:
         NETCONFIG_DNS_STATIC_SERVERS=”yourresolverIP”
    netconfig update
    
    Restart network services:
    /etc/init.d/network eth0 restart
    
    Enter Activation Code:
    yast2 --> 
    Support/Novell Customer Center Configuration
    Perform all online updates (you may need to do this many times):
    yast2 --> Software/Online Update
    Software Repositories:

    To use the online software repositories:
    • yast2 --> Software/Software Repositories
    • Disable Suse-Linux-Enterprise-Server-11 11-0
    • Enable SLES11-Pool
    If installing SLES 11 as ESX/ESXi Guest:

    Insert and connect CD in the ESX Client
    Install VMWare Tools in SLES 11:
    mkdir /media/cdrom
    mount /dev/cdrom /media/cdrom
    rpm -i VMwareTools-4.0.0-164009.i386.rpm
    /usr/bin/vmware-config-tools.pl
    


    Modules for RT:

    Install, using zypper:
    • apache2-mod_perl
    • apache2-mod_python
    • mysql-max
    • gd
    • ncftp
    • openssl
    • perl-DBD-mysql
    • perl-MIME-tool
    Install, using rpm:
    • zlib-devel (latest version):
    wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/Base:/build/standard/x86_64/zlib-devel-1.2.3-135.52.x86_64.rpm
    • libopenssl-devel (version 0.9.8h to match SLES 11 installation):
    ncftpget ftp://rpmfind.net/linux/opensuse/update/11.1/rpm/x86_64/libopenssl-devel-0.9.8h-28.7.1.x86_64.rpm
    Install lynx (latest version) from source:
    wget http://lynx.isc.org/current/lynx-cur.tgz
    Install Perl CPAN modules:
    perl -MCPAN -eshell:
         o conf prerequisites_policy follow
         o conf commit
         install YAML
         install Bundle::CPAN
         reload CPAN
         install LWP
         install Apache::DBI
    
    Install Test::Pod manually (version 1.26; version 1.40 does not test OK on SLES 11) :
    wget http://search.cpan.org/CPAN/authors/id/P/PE/PETDANCE/Test-Pod-1.26.tar.gz
    MYSQL Configuration:
    /etc/init.d/mysql start
    /usr/bin/mysql_secure_installation
    


    RT Installation:

    Create RT user and group:
    groupadd rt
    useradd -m -g rt rt_user
    Download RT:
    wget http://download.bestpractical.com/pub/rt/release/rt.tar.gz
    tar xf rt.tar.gz
    cd rt-3.8.4
    ./configure --prefix=/opt/rt3 --with-web-user=rt_user --with-web-group=rt --with-db-database=rt3 --with-db-rt-pass=yourpass --with-db-type=mysql --with-rt-group=rt --with-apachectl=/usr/local/apache2/bin/apachectl
    
    Fix Perl dependencies (you may need to do this more than once):
    make fixdeps
    Check dependencies:
    make testdeps
    Install RT:
    make install
    make initialize-database
    RT Site Configuration:

    Configure RT_SiteConfig.pm:
    vi /opt/rt3/etc/RT_SiteConfig.pm:
         Set($rtname , "rt.yourdomainename");
         Set($Organization , "yourdomainename");
         Set($CorrespondAddress , 'yourname@yourdomainename');
         Set($CommentAddress , 'yourname@yourdomainename');
         Set($OwnerEmail , 'root');
         Set($MaxAttachmentSize , 10000000);
         Set($WebBaseURL, "http://rt.yourdomainename");
         Set($WebImagesURL , $WebPath . "/NoAuth/images/");
         Set($LogoURL, $WebImagesURL . "yourlogo.gif");
         Set($LogoLinkURL, 'http://yourdomainename');
         Set($LogoImageURL, $WebImagesURL . "yourlogo.gif");
         Set($LogoAltText, "your company name");
    
         # THE DATABASE:
    
         Set($DatabaseType, 'mysql');
         Set($DatabaseUser , 'rt_user');
         Set($DatabasePassword , 'yourpassword');
         Set($DatabaseName , 'rt3');
    
         # THE WEBSERVER:
    
         Set($WebPath , "");
         #Set($WebPath , "/");
         Set($WebBaseURL , "http://rt.yourdomainename");
         1;
    
    Allow web server to write in mason_data directory:
    chown wwwrun.www /opt/rt3/var/mason_data
    chown wwwrun.www /opt/rt3/var/mason_data/obj
    


    Apache2 Configuration:

    Add Perl module:
    vi /etc/sysconfig/apache2:
                     add "perl" to the APACHE_MODULES variable
    
    Create rt.conf virtual host file:
    vi /etc/apache2/vhosts.d/rt.conf:
         <VirtualHost *:80>
               ServerAdmin yourname@yourdomainname
               ServerName rt.yourdomainname
               DocumentRoot /opt/rt3/share/html
               AddDefaultCharset UTF-8
               SetHandler perl-script
               PerlHandler RT::Mason
               PerlModule Apache::DBI
               PerlRequire /opt/rt3/bin/webmux.pl
         <Directory /opt/rt3/share/html>
               Options ExecCGI +FollowSymLinks
               AllowOverride All
               Order deny,allow
               Allow from 192.168.1.0/24  # watch for HTTP/1.1 403 Forbidden
               Deny from all
         </Directory>
         </VirtualHost>
    
    If you get the following warnings in /var/log/apache2:
           apr_sockaddr_info_get() failed:
    vi /etc/hosts:
         yourIP yourhost yourFQDN
    
    Apache2/SSL Configuration:

    Get certificate and key fiels from a CA and put them in:
          /etc/apache2/ssl.crt/server.crt
          /etc/apache2/ssl.key/server.key

    If you need a self-signed certificate instead:
    openssl req -new -x509 -nodes -out server.crt -keyout server.key
    cp server.crt /etc/apache2/ssl.crt/server.crt
    cp server.key /etc/apache2/ssl.key/server.key
    
    Create an SSL site-specific configuration file:
    vi /etc/apache2/vhosts.d/rt-ssl.conf:
    
    <VirtualHost *:443>
    
           ServerAdmin yourname@yourdomain
           ServerName yourhost.yourdomain
       
           DocumentRoot /opt/rt3/share/html
           SSLEngine on
    
           SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
           SSLCertificateFile /etc/apache2/ssl.crt/server.crt
           SSLCertificateKeyFile /etc/apache2/ssl.key/server.key
    
           <Files ~ "\.(cgi|shtml|phtml|php3?)$">
               SSLOptions +StdEnvVars
           </Files>
    
           <Directory "/srv/www/cgi-bin">
               SSLOptions +StdEnvVars
           </Directory>
    
           SetEnvIf User-Agent ".*MSIE.*" \
                    nokeepalive ssl-unclean-shutdown \
                    downgrade-1.0 force-response-1.0
    
           CustomLog /var/log/apache2/ssl_request_log   ssl_combined
    
           AddDefaultCharset UTF-8
           SetHandler perl-script
           PerlHandler RT::Mason
           PerlModule Apache::DBI
           PerlRequire /opt/rt3/bin/webmux.pl
    
           <Directory /opt/rt3/share/html>
                   Options ExecCGI +FollowSymLinks
                   AllowOverride All
                   Order deny,allow
                   Allow from 192.168.1.0/24
                   Deny from all
           </Directory>
    
    </VirtualHost>
    
    To force apache2 to load with SSL at boot-time, comment out the following tags:

          <IfDefine SSL>
          <IfDefine !NOSSL>
          </IfDefine>
          </IfDefine>

    in the following files:
          /etc/apache2/ssl-global.conf
          /etc/apache2/listen.conf



    RT Mail Configuration:

    Create an alias that will accept mail and post it in a queue:
    vi /etc/aliases:
         youralias: "|/opt/rt3/bin/rt-mailgate --queue 'yourqueuename' --action correspond --url http://yourhost.yourdomain"
    newaliases
    
    Make sure your queue has at least assigned "CreateTicket" rights to the people who will send to it.

    If you are going to fetchmail from another server:
    vi /etc/fetchmailrc:
         poll "yourotherserver" protocol yourpostofficeprotocol : user "youruser@yourdomain" with password "yourpassword" is "yourqueuename" here ;
    
    Final Steps:

    Perform all updates:
    yast2 --> Software/Online Update
    Make sure services load at boot-time:
    chkconfig mysql on
    chkconfig apache2 on
    chkconfig -d smartd
    
    Optional/Suggested:
         install nmap and remove any unnecessary services with zypper

    Restart server to see if everything loads correctly at boot time:
    shutdown -r now

    jeudi 3 mars 2011

    Fou... l'enseignant ou l'étudiant?

    Une imitation, plus ou MOINS réaliste, de moi quand je parle de Linux dans mes cours. Doit-on s'inquiéter de la relève en informatique? ;)

    mardi 1 mars 2011