carbon-emacs-icon.png

Emacs

This is my page devoted to Emacs. You will find here especially my tarball of Emacs 64bits for Windows.

Why Emacs?

Well, I'm probably biased on this one.

Emacs is the first and oldest programmable editor still in use and it is actively developed. Moreover it is Lisp based, which ensures a high flexibility.

I have met Emacs somewhere around 1985 and I couldn't truly do any programming without it.

I have used vi, which was at the lowest stage in the … of the text editors species. Vim has become a contender in the arena of programmable text editors only recently. There are others that are certainly very valuable too like Sublime Text(TM). But once you look at something like Org mode for Emacs, either you get convinced … or disgusted for the eternity (but in this case, Emacs is really not for you!)

Other programmable development environments like the MS Visual Studio editor are targeted at … programming (not really strange !) I think is more efficient than Emacs if you do C++/C#, especially in the Windows environment. Don't talk Eclipse to me … I never adopted its ways of running things even if I acknowledge that it helps a lot when doing Java (but anyway who wants to program in Java?).

Moreover none of those programming environments take Lisp development in charge, and certainly not at the level the Slime mode does. So given that I'm still a Common Lisp addict, I'm very happy with Emacs.

Plus, Org mode and AUC-TEX are unavoidable for anybody fed up with braindead GUI apps that allways do half of the job. Typesetting and programming is always a matter of handling text input (not mouse input !) so a (programmable) text editor is the universal tool for my job.

I have followed Emacs development for a long time mainly because I used it for myself, so I wanted to submit any fixes or enhancements into the main trunk.

I remember a good time with the development of Gnus, the Gnu Emacs newsreader. Lars Magne Ingebregtsen started an amazing piece of code, merging your mailboxes with newsgroups and other sources of information. At this time, newsgroups where the de facto standard to exchange information. There was no Google+ or other fancy news sites, blogs or whatever. It predates even the Web (am I that old? :-))

I found XEmacs to be much more appealing than Gnu Emacs and at some point, there has been a huge investment around a true multi-platform XEmacs, thanks to a couple (well probably a dozen actually) of guys which did an amazing job, Ben Wing being probably the most prolific one. So at that time I was using XEmacs under Windows, because it provided features Gnu Emacs couldn't, like the ability to display any kind of glyph in the buffer, and even controls like checkboxes.

Unfortunately, XEmacs came short of these achievements and development get stuck. There has been a problems with the FSF, GPL, parts of the code belonging to Sun Microsystems, whatever. You can find the whole story on the XEmacs web site. The net result was that at some point, Gnu Emacs managed to provide the main features that made XEmacs superior. That was version 23 of Gnu Emacs IIRC. Given the good support of the Windows platform, I switched – reluctantly at first – to Gnu Emacs and I'm very happy with it now.

With the advent of 64-bits versions of Windows and laptops with enough RAM to support it, I started to think that I could benefit from a large Emacs, a 64-bits one for Windows. The job had already been done for Linux, so it was really a matter of extending it to the Windows case. But you can bet that in this area, not that much of the underlying code is shared between platforms. Moreover, I used to compile Emacs with the MS C compiler. You may wonder why using such a strange beast to compile Gnu Software? Well, if you look at my other page about TEX, you'll get the reasons to use the MS C compiler rather than GCC. So I had to patch numerous locations where sign extension of 32-bits values to 64-bits values had to be avoided. All in all, it worked pretty well, and I had the help and support of Eli Zaretskii which was very much appreciated.

That is where I am today as of 2013, October.

64 bits emacs for Windows

Download and installation

This is my slightly enhanced version of GNU Emacs for 64-bits Windows.

I stopped compiling Emacs with the MS C compiler. There had been reasons of doing it in the past, but I think that today, you lose much more than you win.

  • Cygwin was never an option to me (see Why Cygwin sucks).
  • GCC had suffered from bugs: optimizing code resulted in failures, whereas the MS C compiler is rock solid for a longer time

However, today MinGW provides a GCC that seems to be solid enough for Emacs and LUATEX compilation. GCC is able to compile DLLs of other projects that Emacs can use : rsvg, libxml2, gnutls, etc. which you wouldn't been able to compile using the MS C compiler. So by using the MinGW GCC + MSYS tools, you get the better of both worlds: native WIN64 executables, fully loaded with all available features.

What you get in this version :

  • Emacs trunk as of 2014-05-20
  • compiled with optimizations and stripped
  • with patches not yet published
  • with enhanced emacsclient from EmacsW32
  • with all dlls supported by Emacs: Xpm, LibPNG, LibJPEG, LibTIFF, LibGIF, LibRSVG, LibGDX_Pixbuf, LibGObject, LibGNUTLS, LibXML2, ZLib.

Preview of emacs-24.4+ as of bzr117129.

Previous versions:

The dlls provided are the one from the MSYS2/MinGW64 project. I haven't recompiled all of these dlls. I'm using them myself and they seem reliable. Should you encounter any problem, please warn me about it.

The version of emacsclient provided is not the GNU one, but the one from EmacsW32. The reason for this is that you can start Emacs from this Emacsclient, which avoids a spurious console. You can download the source code for this emacsclient, revisited by me. It would need a full rewrite IMHO. I may revert to use the provided emacsclient, as it has been enhanced and supports features of recent Windows versions that the one from EmacsW32 doesn't.

There is no installer. All you have to do is:

  1. Unpack in a location like c:\Local\
  2. Setup a shortcut on your desktop to c:\Local\Emacs\bin\emacsclientw.exe
  3. Double click the new shortcut.

Updates

Status as of 2014-05-09

My current patch addresses two points:

  1. mmap'ed buffers for Windows
  2. malloc for Windows

Memory mapped buffers have been left in a half-baked stated. So I took the liberty of cleaning up things that are not used. There is also no need for the mmap_set_vars() function: buffers have to be reallocated when emacs starts. We don't care about dumping previous buffers addresses, since they will be meaningless when emacs will restart after dump anyway. Unless it is considered as a security hole to dump such adresses, but in this case, are we sure not to dump other such addresses anyway?

So the first part is to simplify a little bit these mmap'ed buffers handling.

Second part is about adding mmap'ed buffers to Windows and to change malloc support. Adding mmap'ed buffers resumes to adding mmap_alloc(), mmap_realloc() and mmap_free() functions. They are much simpler than the Unix couterparts because of the availability of VirtualQuery() API which allows to query an address about the virtual block it belongs to. So no need to keep track of the allocated blocks. In the current state, mmap'ed buffers are very inefficient in case you need to grow your buffer, like it is the case when you visit a compressed file. The buffer is constantly reallocated, one page at a time, and it takes forever. I solved this problem by doing amortized allocation: twice the needed memory is reserved, and pages committed when needed. This makes the mmap'ed buffers almost as fast as the previous (Gnu malloc + relocatable allocator) scheme. It is also safer (seems ralloc.c is not that much). It puts less pressure on virtual memory, because we don't reserve a single huge block at first.

Changing malloc to switch to native heap allocation is more challenging. It requires to reconsider the way dumping is done. The biggest problem is that we can't dump the heap data structure itself with the hope to reuse it when emacs is run. This works with gmalloc because the data structure is exposed and known. Once we rely on the Windows heap data strctures, it is no more the case. Actually, the problem is that there is no guarantee that the heap data structure doesn't change from one version of Windows to another. It has changed when the Low Fragmentation Heap has been introduced in Vista. So to simplify everything, the trick is to admit that we don't care to free what has been dumped: a big part of it is constant, and some objects that should be freed won't be. Before dump, we setup a heap into a static array. This array will automatically be dumped with the executable, so all addresses inside this array will be valid when emacs will be restarted. At this point, we use a regular system heap. When freeing memory area, we do it only if it is outside the dumped data, and the same happens for reallocation.

Last, before dump, we use a non-growable heap that can handle objects only up to some maximum size. So we need to take care of objects larger than this. Currently, those objects are all garbage collected before dumping so that shouldn't be a problem. However, Emacs GC is conservative, and problems may happen. See http://lists.gnu.org/archive/html/emacs-devel/2014-05/msg00270.html . So to take care of such large blocks, we implement a very basic first-fit allocation algorithm using the end of the static data array.

The greatest benefit of this new malloc scheme is that there is no need to add a new data segment for the dumped data. The emacs binary can be stripped. The whole building process is much simpler. The dumped data occupy approximately 18Mb for 64bits and 10Mb for 32bits.

I have run some tests.

Changing malloc() to rely on the system heap allocator has no measurable impact. Allocating several hundred Mb by consing a huge list does not show any measurable penalty or improvement. The runtime stays within a few percents of what it is by using gmalloc. Sometimes slower, sometimes faster.

The elisp code below:

(require 'cl)

(defun select-random (l)
  (nth (random (length l)) l ))

(defun select-random-point ()
  (+ 1 (random (buffer-size))))

(defun create-random-string-size (size)
  (make-string size (+ (random 64) 32)))

(defun stress-test ()
  (let (buffers
        strings)
    (dotimes (i 50)
      (push (switch-to-buffer (symbol-name (gensym "buf"))) buffers))
    (dotimes (i 100000)
      (push (create-random-string-size 10000) strings))
    (dolist (buf buffers)
      (with-current-buffer buf
        (insert (select-random strings))))
    (dotimes (i 1000000)
      (if (evenp (random 2))
          (with-current-buffer (select-random buffers)
            (goto-char (select-random-point))
            (insert (select-random strings)))
        (with-current-buffer (select-random buffers)
          (let ((p1 (select-random-point))
                (p2 (select-random-point)))
            (delete-region (min p1 p2) (max p1 p2))))))
    (dolist (buf buffers)
      (let (kill-buffer-query-functions)
        (kill-buffer buf)))
    (garbage-collect)))

(stress-test)

(kill-emacs)

does some trashing into buffers. Actually, it heavily trashes memory. You can try it and compare with a regular emacs.

I have discarded many of my source patches. Emacs doesn't support the MS C compiler anymore. I think it is a pity, but emacs developers do everything they can in this direction. Most of my other patches have been integrated into the trunk by Eli Zaretskii. So this version is ahead of trunk only by:

  • use of mmap'ed buffers (so no relocatable allocator anymore, mmap is much faster)
  • dumped data are stored into bss segment instead of an extra segment (allows to strip the executable)
  • emacsclient from EmacsW32 by Lennart Borgman, a little bit revamped by me.

Patches to GNU Emacs 24.4 bzr117129

Older versions

I started with the idea to comment all my attempts to enhance Emacs, but unfortunately, things have become too messy for that. So I'll sketch what I'm doing and I pray that in the future, things will become simpler (in the hands of Eli Zaretskii and whoever wants to go along these lines).

Here is the link to the patch as of 2014-02-05.

  1. The multi-platform multi-compiler nightmare. It is assumed that Gnu Emacs can be compiled under 32-Bits and 64-bits of Windows, with both the MS C compiler and GCC. Worse than that, the FSF wants to keep support of old platforms and compilers, even under Windows. This habit is a Gnu/Linux habit where people are used to download a source tarball and compile it on their system. Under Windows, people are used to download a binaries tarball and don't want to mess up with this compilation process. (It seems that the FSF has never understood that!) So in order to simplify things, I started to cleanup the following :
    • use system headers whenever it is possible, rather than provided replacements. This led to split the nt/inc directory: one for GCC and one for MS. At least it is a first step in knowing what is useful to whom.
    • the previous rule is broken in several places for 'old' reasons. I don't think it is a good practice. Problems should be fixed at the right place, so if there is a need to replace a system header, that is because the code is wrong somewhere else.
    • One of the byproducts of this is that I had to rename the src/process.h file into src/eprocess.h: guess what? MS is providing such a system header file, which conflicts with the fact that Emacs build system adds ='-I$(SRC)'= to every GCC command. Well, that is only one of the bad decisions (well, unknown to be bad at first, but never fixed) that continues to spoil Emacs source code when you want to compile it under Windows with MS tools.

    The patch I give has been tested only for the MinGW64+MSYS platform. However, I compiled some previous version with both MingW32 and MSVC2012. I don't have time to always check that it is still working.

  2. I changed the layout. Under Windows, I find it much easier to make 2 versions of Emacs coexist by storing them in full different directories. The default layout is once again very Gnu/Linux oriented, hence the changes in aclocal.m4 .
  3. I added mmap() support for buffers, which was surprisingly easy.
  4. I changed the way elisp code is dumped into the executable. The standard way adds a new segment to the executable, but this new segment prevents to strip the executable and some other things. Given the fact that Emacs knows already how to dump its own static data, I did the same for the elisp code. It was easy to declare a byte array large enough and store those byte codes there. The net result after that: the Emacs executable is not crippled by the supplementary data segment, it can be stripped and be investigated by all COFF (MS or Gnu) tools, which was not the case before. Something useful to do in this area would be to import the portable dumper of XEmacs into Gnu Emacs, but it is quite an undertaking. BTW, their generational garbage collector would be nice too!
  5. I fixed an issue with ACLs by switching the Win32 API call to another one which seems to actually work. Otherwise in certain situations (not infrequent actually: happened all the time for me), Emacs kept getting error codes that had to be ignored, because the call was failing.

Whoever wants to sort this out and submit patches to the FSF based on this work is free to do it. I stopped fighting with the FSF because my prerequisites before submitting the interesting parts of this were to fix the build process. Eli Zaretskii did an amazing job with autoconf and stuff, but my point was not there. It was about sorting the header include mechanism: as-is, the trunk never compiled out of the box for me, neither with the MS C compiler, nor with MinGW64. I tried several times, but could never go through to a point where it would be 'clean' enough. There has been reluctance to change things at several places, with reasons that were contradictory. So I stopped by lack of time and energy to devote to this.

The configuration

Here is how I configured Emacs :

#!/bin/sh
CFLAGS="-I/mingw64/include -O3 -g" CPPFLAGS="-I/mingw64/include" LDFLAGS="-L/mingw64/lib" ./configure --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --prefix=/c/Local/Emacs --libexecdir=/c/Local/Emacs/bin --datarootdir=/c/Local/Emacs --localstatedir=/c/Local/Emacs --sysconfdir=/c/Local/Emacs/etc --with-jpeg --with-xpm --with-png --with-tiff --with-rsvg --with-xml2 --with-gnutls

I have adopted a non-standard layout for the directories, because I think that under Windows, the GNU/Linux traditional one doesn't make a lot of sense. I don't like at all the idea of mixing several versions of the same program into the same directory tree, because the only way to remove one of them (for example) is to rely on some Makefile that did the job of installing them. I vastly prefer to be able to move or delete the whole subtree with only one command. So no sharing. It has also the advantage to remove a couple of deep directories, so the tree is shallower.

Here is a part of the config.log file if you ever want to check :

## ----------- ##
## confdefs.h. ##
## ----------- ##

/* confdefs.h */
#define PACKAGE_NAME "GNU Emacs"
#define PACKAGE_TARNAME "emacs"
#define PACKAGE_VERSION "24.4.50"
#define PACKAGE_STRING "GNU Emacs 24.4.50"
#define PACKAGE_BUGREPORT "bug-gnu-emacs@gnu.org"
#define PACKAGE_URL "http://www.gnu.org/software/emacs/"
#define PACKAGE "emacs"
#define VERSION "24.4.50"
#define MAIL_USE_POP 1
#define STDC_HEADERS 1
#define HAVE_SYS_TYPES_H 1
#define HAVE_SYS_STAT_H 1
#define HAVE_STDLIB_H 1
#define HAVE_STRING_H 1
#define HAVE_MEMORY_H 1
#define HAVE_STRINGS_H 1
#define HAVE_INTTYPES_H 1
#define HAVE_STDINT_H 1
#define HAVE_UNISTD_H 1
#define __EXTENSIONS__ 1
#define _ALL_SOURCE 1
#define _DARWIN_C_SOURCE 1
#define _GNU_SOURCE 1
#define _POSIX_PTHREAD_SEMANTICS 1
#define _TANDEM_SOURCE 1
#define _FILE_OFFSET_BITS 64
#define _DARWIN_USE_64_BIT_INODE 1
#define SYSTEM_TYPE "windows-nt"
#define HAVE_PWD_H 1
#define HAVE_SYS_SOCKET_H 1
#define HAVE_STDLIB_H 1
#define HAVE_UNISTD_H 1
#define HAVE_SYS_PARAM_H 1
#define HAVE_PTHREAD_H 1
#define HAVE_DIRENT_H 1
#define HAVE_SYS_TIME_H 1
#define HAVE_WCHAR_H 1
#define HAVE_STDINT_H 1
#define HAVE_INTTYPES_H 1
#define HAVE_SYS_STAT_H 1
#define HAVE_UTIME_H 1
#define TIME_WITH_SYS_TIME 1
#define HAVE_DECL_SYS_SIGLIST 0
#define HAVE_DECL___SYS_SIGLIST 0
#define HAVE_SYS_WAIT_H 1
#define HAVE_LONG_FILE_NAMES 1
#define HAVE_NTGUI 1
#define SYSTEM_MALLOC 1
#define HAVE_GETPAGESIZE 1
#define USE_MMAP_FOR_BUFFERS 1
#define HAVE_RSVG 1
#define HAVE_GNUTLS3 1
#define HAVE_GNUTLS 1
#define HAVE_W32NOTIFY 1
#define USE_FILE_NOTIFY 1
#define USE_TOOLKIT_SCROLL_BARS 1
#define USE_XIM 1
#define HAVE_XPM 1
#define HAVE_JPEG 1
#define HAVE_JPEG 1
#define HAVE_ZLIB 1
#define HAVE_PNG 1
#define HAVE_TIFF 1
#define HAVE_GIF 1
#define HAVE_LIBXML2 1
#define HAVE_H_ERRNO 1
#define HAVE_GETHOSTNAME 1
#define HAVE_RANDOM 1
#define HAVE_RINT 1
#define HAVE_SELECT 1
#define HAVE_GETPAGESIZE 1
#define HAVE_SETLOCALE 1
#define HAVE_SHUTDOWN 1
#define HAVE_SETITIMER 1
#define HAVE_SENDTO 1
#define HAVE_RECVFROM 1
#define HAVE_GETSOCKNAME 1
#define HAVE_GETPEERNAME 1
#define HAVE_COPYSIGN 1
#define HAVE_LOG2 1
#define HAVE___BUILTIN_UNWIND_INIT 1
#define HAVE_FSEEKO 1
#define HAVE_TZSET 1
#define HAVE_READLINKAT 1
#define HAVE_FACCESSAT 1
#define HAVE_FCNTL 1
#define HAVE_FSTATAT 1
#define HAVE_FSYNC 1
#define HAVE_GETTIMEOFDAY 1
#define HAVE_LSTAT 1
#define HAVE_MKOSTEMP 1
#define HAVE_PIPE2 1
#define HAVE_PSELECT 1
#define HAVE_PTHREAD_SIGMASK 1
#define HAVE_READLINK 1
#define HAVE_STRTOIMAX 1
#define HAVE_STRTOUMAX 1
#define HAVE_SYMLINK 1
#define LOCALTIME_CACHE 1
#define HAVE_INET_SOCKETS 1
#define vfork fork
#define HAVE_SNPRINTF 1
#define HAVE_LANGINFO_CODESET 1
#define HAVE_MBSTATE_T 1
#define __restrict_arr __restrict
#define HAVE_SOCKETS 1
#define NULL_DEVICE "NUL:"
#define SEPCHAR ';'
#define subprocesses 1
#define USER_FULL_NAME pw->pw_gecos
#define DIRECTORY_SEP '/'
#define IS_DEVICE_SEP(_c_) ((_c_) == ':')
#define IS_DIRECTORY_SEP(_c_) ((_c_) == '/' || (_c_) == '\\')
#define IS_ANY_SEP(_c_) (IS_DIRECTORY_SEP (_c_) || IS_DEVICE_SEP(_c_))
#define DEFAULT_SOUND_DEVICE "/dev/dsp"
#define GC_SETJMP_WORKS 1
#define DOS_NT /**/
#define WINDOWSNT 1
#define USABLE_FIONREAD 1
#define COPYRIGHT "Copyright (C) 2014 Free Software Foundation, Inc."
#define TERM_HEADER "w32term.h"
#define EMACS_CONFIGURATION "x86_64-w64-mingw32"
#define EMACS_CONFIG_OPTIONS "--host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --prefix=/c/Local/Emacs --libexecdir=/c/Local/Emacs/bin --datarootdir=/c/Local/Emacs --localstatedir=/c/Local/Emacs --sysconfdir=/c/Local/Emacs/etc --with-jpeg --with-xpm --with-png --with-tiff --with-rsvg --with-xml2 --with-gnutls 'CFLAGS=-I/mingw64/include -O3 -g' CPPFLAGS=-I/mingw64/include LDFLAGS=-L/mingw64/lib"
#define config_opsysfile <ms-w32.h>
#define HAVE_ALLOCA 1
#define HAVE_UNSIGNED_LONG_LONG_INT 1
#define HAVE_C99_STRTOLD 1
#define HAVE_ENVIRON_DECL 1
#define HAVE_DECL_FDATASYNC 0
#define HAVE_DECL_STRMODE 0
#define LSTAT_FOLLOWS_SLASHED_SYMLINK 1
#define __GETOPT_PREFIX rpl_
#define HAVE_DECL_GETENV 1
#define restrict __restrict
#define HAVE_LONG_LONG_INT 1
#define HAVE_DECL_MEMRCHR 0
#define HAVE_DECL_ALARM 1
#define HAVE_WCHAR_T 1
#define HAVE_DECL_TZNAME 1
#define HAVE_TZNAME 1
#define HAVE_DECL_STRTOIMAX 1
#define HAVE_DECL_STRTOUMAX 1
#define HAVE_DECL_UNSETENV 1
#define HAVE_STRUCT_UTIMBUF 1
#define GETGROUPS_T int
#define HAVE_CLOCK_GETTIME 1
#define HAVE_CLOCK_SETTIME 1
#define GNULIB_CLOSE_STREAM 1
#define HAVE_DUP2 1
#define GNULIB_FACCESSAT 1
#define HAVE_FDATASYNC 1
#define HAVE_DECL_FDOPENDIR 0
#define GNULIB_FDOPENDIR 1
#define PENDING_OUTPUT_N_BYTES fp->_ptr - fp->_base
#define __GETOPT_PREFIX rpl_
#define GETTIMEOFDAY_TIMEZONE struct timezone
#define GNULIB_MKOSTEMP 1
#define HAVE_SYS_ACL_H 1
#define HAVE_ACL_GET_FILE 1
#define HAVE_ACL_SET_FILE 1
#define HAVE_ACL_FREE 1
#define HAVE_ACL_FROM_TEXT 1
#define USE_ACL 1
#define GNULIB_FSCANF 1
#define GNULIB_SCANF 1
#define my_strftime nstrftime
#define _GL_WINDOWS_64_BIT_ST_SIZE 1
#define nlink_t int
#define HAVE_DECL_LOCALTIME_R 0
#define HAVE_UNSETENV 1
#define HAVE_WINDOW_SYSTEM 1
#define EMACS_CONFIG_FEATURES "XPM JPEG TIFF GIF PNG RSVG NOTIFY ACL GNUTLS LIBXML2 ZLIB"

configure: exit 0

Stuff to investigate

DONE Graphene

Graphene seems quite nice.

DONE Prelude

I have started to rebuild my .emacs.d configuration on top of Prelude and I have added parts of Graphene. This is still work in progress.

Problems with Windows

Autohotkey doesn't work for modifiers remapping

Need to do it using the registry and some fancy app like http://sharpkeys.codeplex.com/

Taskbar doesn't autohide when emacs (or other app) is fullscreen

Actually, it hides itself, but doesn't autoshow. Curiously, it autoshows on secondary screens, but not on the primary one.

https://connect.microsoft.com/IE/feedback/details/808743/auto-hide-funcition-not-functional-in-fullscreen-mode-ie11-for-windows-7#tabs