This coming weekend I’ll be at the Renegade Craft Fair in Brooklyn! It’s been a few years since I’ve been to New York, and I’m really pleased to have the chance to be back.
I MAY BE IN A BOOTH THAT LOOKS SOMETHING LIKE THIS:
And I will be super happy to say hello to you! And even — IF YOU LIKE — make you a Roll-a-Sketch or something.
I have also heard unconfirmed rumors that I may have something very new and exciting at this show:
My (Eisner Award-nominated) book Beards of our Forefathers, originally a hardcover that is now out of print, has just been reissued in a super-snazzy paperback edition! It’s not for sale online yet, but I hope to have a few ADVANCE COPIES in Brooklyn. Fingers crossed.
After that: San Diego Comic-Con in July; Gen Con in August. I’ve been working very very hard to get the Machine of Death card game off to the printers, and I think this week is the week! I think this week might be the week I finally cross the last T and dot the last I and get that whole thing finally, finally done, after a labor of many months. I can’t wait!!
This is a special-purpose release to match the arrival of “Realm of Racket” in bookstores. Racket v5.3.5 adds a single realm collection to the v5.3.4 release. The new collection contains the source code that readers of Realm may wish to use for experiments.
Today’s exercise is a classic problem of computer science: given an array of positive and negative integers, find three that sum to zero, or indicate that no such triple exists. This is similar to the subset sum problem, which we solved in a previous exercise, but simpler because of the limit to three items. A brute force solution that operates in O(n3) time uses three nested loops to select items from the array and test their sum, but an O(n2) solution exists. For instance, in the array [8, -25, 4, 10, -10, -7, 2, -3], the numbers -10, 2 and 8 sum to zero, as do the numbers -7, -3, and 10.
Your task is to write a program that solves the 3SUM problem in O(n2) time. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.
Among the 4 Lisp’s equality operators equalp may be the least used, but it has some really cool features:
it matches strings ignoring case (this is well-known, I suppose)
it also matches structs
it also matches vectors and arrays
and it matches hash-tables
The last 3 are really useful, because unlike strings and other primitive types, there’s no specialized equality predicates for built-in structural types, so equalp fills this role.
Over at What keramida said…, Giorgos Keramidas poses an interesting problem. Given the following output from dstat, get rid of the B, k, and M suffixes by converting them into bytes.
Keramidas also wants to do a bit of reformatting of the table, which we’ll ignore for this post. He does the conversion with three invocations of replace-regexp but given that he titles his post Powerful Regular Expressions Combined with Lisp in Emacs, I thought it would be an interesting challenge to write some Elisp to get rid of those suffixes and convert the entries into bytes.
Obviously, for a one-off task, Keramidas’ solution is the best but if this is something that has to be done regularly a bit of Elisp is just what’s needed. My solution is a Lisp function that converts the table with a single call. Leave your solution in the comments and I’ll post mine in a few days.
Problem: Determine if a number is prime, with an acceptably small error rate. Solution: (in Python) Discussion: This algorithm is known as the Miller-Rabin primality test, and it was a very important breakthrough in the study of probabilistic algorithms. Efficiently testing whether … Continue reading →
Every Emacs user knows that he can split the current window
horizontally (with C-x 2) and vertically (with C-x 3) as much as
he desires to. However, some Emacs users don’t know what to do with
the extra windows they’ve created when they do not them.
To delete the selected window, type C-x 0 (delete-window).
Once a window is deleted, the space that it occupied
is given to an adjacent window (but not the minibuffer window, even if
that is the active window at the time). Deleting the window has no effect on the
buffer it used to display; the buffer continues to exist, and you can
still switch to with C-x b or any other buffer navigation command.
C-x 4 0 (kill-buffer-and-window) is a stronger (and fairly
unknown) command; it kills the current buffer and then deletes the
selected window (basically it combines C-x k and C-x 0). Obviously
it’s a good idea to use it on windows displaying buffers you’re no
longer needing.
C-x 1 (delete-other-windows) deletes all the windows, except the
selected one; the selected window expands to use the whole frame.
(This command cannot be used while the minibuffer window is active;
attempting to do so signals an error.) In the era of narrow screens I
used that command fairly often when I needed to focus on a particular
task. Now I keep my screen split in half vertically 99% of the time,
but I still use C-x 1 from time to time when I’m about to resplit my
screen in some uncommon way.
Windows displaying help buffers (generally created with commands like
C-h ...) warrant a special mention. They can be deleted with a
single keystroke - q. That would delete the help window altogether
if it was created by the help command, or restore its original
content if the window existing beforehand and was reused by the help command.
This post continues a topic that was introduced in
smarter open-line few months
back.
Often when editing code one wishes to open a line just above the
current one, which is properly indented relative to the existing code,
and position the cursor at its beginning. Such a feature is present
in most IDEs, such as IntelliJ IDEA, Eclipse and NetBeans. It’s
sometimes bound to Control+Shift+Enter. Last time I showed you how
to implement a similar function called smart-open-line, this time
will implement smart-open-line-above. Just add this snippet to your
.emacs (or .emacs.d/init.el or whatever):
12345678910
(defunsmart-open-line-above()"Insert an empty line above the current line.Position the cursor at it's beginning, according to the current mode."(interactive)(move-beginning-of-linenil)(newline-and-indent)(forward-line-1)(indent-according-to-mode))(global-set-key[(controlshiftreturn)]'smart-open-line-above)
Evaluate the code (or restart Emacs) and you’ll be able to use
M-x smart-open-line-above or Control+Shift+Enter (aka C-S-return).
Admittedly this keybinding kind of sucks, so here’s another option for
you - M-o (used by default as the prefix for some font setting
commands nobody ever uses) for smart-open-line and M-O for
smart-open-line-above.
commander.el is a command
line parser for Emacs. It avoids messing with command-switch-alist
(and friends) and instead defines the schema in an elegant API.
Example schema
Here is a (silly) example where numbers can be added and subtracted.
The ratio of the circumference to the diameter of a circle, known as π, is constant regardless of the size of the circle; that fact has been known to mathematicians for about five thousand years; much of the history of mathematics is intertwined in the history of πi, as approximations of the ratio have improved over time. Much of the history of this blog is also intertwined in the calculation of π; one of the original ten exercises used a bounded spigot algorithm of Jeremy Gibbons to calculate π, and later we used an unbounded spigot algorithm also due to Gibbons; we studied the ancient approximation of 355/113 calculated by Archimedes; we studied twodifferent Monte Carlo simulations of π; and we even had the Brent-Salamin approximation contributed by a reader in a comment.
The development of computers allows us to compute the digits of π to an astonishing accuracy; the current record, unless somebody has bettered it recently, is ten trillion digits. That record was set by the Chudnovsky brothers, two Russian mathematicians living in New York, using an algorithm they developed in 1987. The algorithm is based on a definition of π developed by Ramanujan, and is beautifully described by the two brothers.
Your task is to compute many digits of π using the Chudnovsky algorithm. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.
ELS 2013, the 6th European Lisp Symposium, took place in Madrid on June 3th and 4th.
During the symposium, Nick Levine recorded audio versions of the presentations and collected them here. Nick has also agreed to collect other material from the presenters (slides notably) and put it at the same location. The page is growing up as we speak.
It was good to see you all in Madrid. Thanks to all the presenters and thanks Nick for doing this!
Most people that use Emacs are programmers - that’s a fact!
Most programmers have to deal with relational databases - that’s also a fact!
Programmers often interact with the database they have to use via some
dedicated client - be it a console or a GUI one. One fairly little
known fact about Emacs is that it features a mode that wraps around
terminal database clients, called SQLi. The mode allows you to
interact with a relational database from the comfort of Emacs. Sure,
you can run a terminal client like psql directly from ansi-term
(for instance) as well - but if you’re using a client with no
readline support (like the default clients for Oracle and DB2)
you’ll certainly appreciate SQLi.
Let’s play a bit with SQLi. To create a PostgreSQL connection start
by running M-x sql-postgres. You’ll be prompted for username, database,
password and host and then you’ll be dropped in a buffer dedicated to the
connection you’ve specified. Apart from being able to run all sorts of
SQL in that buffer you’ll also be able to send to it SQL from .sql
files you’re editing in Emacs.
Let’s see how you can do that in a bit more detail. First you have to
associated an SQL file with a connection. While in some .sql file
execute M-x sql-set-product and type postgres. Afterwards do M-x
sql-set-sqli-buffer and select the name of the connection buffer you
want to use (it’s probably called *SQL* if you have only one
connection buffer). Now you’ll be able to use commands like
sql-send-region (C-c C-r) from the .sql buffer and the code from
the region will be executed in the associated connection buffer.
Have a look at the documentation of sql-mode for further details.
Marc Battyani is one of the people who put Lisp at the foundation of their business, and he doesn't seem to regret that decision. And the business itself is exploring very interesting aspects of high-performance and reconfigurable computing. Besides, he has a notable open-source contribution with cl-pdf/cl-typesetting libraries (which even I use, so thanks Marc!). He elaborates on all these and more in much detail in the interview.
Tell us something interesting about yourself.
I think maybe the most unusual things I do is that I work on very different application domains which are even sometimes completely at opposite extremes both in electronics and software. For instance form ultra low power smart sensors based on $1 microprocessors which will run continuously for 5+ years only powered by a small coin battery up to the world's lowest latency supercomputers based on FPGA costing thousands of $ per chip. On the software side I use programming languages ranging from the lowest level languages like assembly or even below with VHDL up to really high level languages like Common Lisp.
I've always enjoyed mixing electronics design and higher level computer science and all that diversity probably gives me a different and original view on say computing and programming in general.
What's your job? Tell us about your company.
I'm the CTO of NovaSparks a startup I founded in 2008 to make ultra-low latency FPGA based supercomputers for the financial markets. BTW These things are really incredibly fast. For instance on 10Gb/s Ethernet market data packets coming from exchanges like the NASDAQ we process the IP/UDP/multicast network stack, extract the messages from the packets, parse/decode/filter/normalize those messages, maintain the indexed order book data structures, aggregate the price levels per stock, generate output messages and finally send them to a server through PCI-express or 10Gb/s Ethernet network stacks. The nice thing is that we do all that fully pipelined at a rate of one message every 12 nanoseconds! To have an idea of how fast it is, in 12 ns the light will only travel 3.6 meters (11.8 ft). Another way to view this performance is that the system can process 83 Millions of financial messages per second without any queuing.
As an aside it is interesting to note that the Domain Specific Language Compilers and various other tools written in Common Lisp have been key enabling factors for the creation of NovaSparks.
I'm also the CEO and CTO of Fractal Concept which is the company where we were developing that technology before starting NovaSparks but as NovaSparks has been using more than 100% of my time for the last years Fractal Concept has been less active with only the development and maintenance of the smart sensors going on.
Do you use Lisp at work? If yes, how you've made it happen? If not, why?
I've always used Common Lisp for most of my work and even when I need to use other programming languages (like VHDL for NovaSpark's FPGAs or javascript for web stuff) I generally use Common Lisp at least for a lot of related tasks like prototyping, designing and testing algorithms, extracting statistics, performing simulations, generating test data, analysing test runs.
The next step is very often to generate some or all the code in other languages by designing various domain specific languages (DSL) which will take care of the tedious aspects of programming in less powerful languages. I really like it when from a few 100s of lines written in a easy to use high level DSL we generate 10000 to 60000+ lines of very low level VHDL code saving months of development.
About that point I think I would even say that I'm mostly doing Language Oriented Programming by trying to abstract the domain specific knowledge into various domain specific languages with very different syntaxes like s-expressions, C like or other languages or even sometimes adding to the mix some GUI or data-based inputs. Then those DSLs would generate most if not all the application code in whatever language is needed be it VHDL, asm, C, javascript or Common Lisp.
What brought you to Lisp? What holds you?
A friend of mine gave me a version of Le_Lisp a version of Lisp used in the 80's by the French universities and engineering schools to teach high level programming concepts. At that time I was mostly programming in Z80 assembly language on a TRS80 and the straight jump from ASM to Lisp was quite a shock and an eye opener.
Since then I've used Common Lisp for countless projects ranging from a few hours of work to multi-years ones and I still find it awesome. Where else can you find a language providing such powerful and multi-paradigms features like CLOS, generic functions, the MOP, macros, closures, s-expressions, lambdas, an interactive REPL (Read Eval Print Loop), native compilation of applications, on the fly native compilation of generated code, the condition system, interactive live programming, real time live debugging of running software and more!
What's the most exciting use of Lisp you had?
I've got so many of them than picking only one would be difficult so here are a few of them:
softscan: real time driving and data acquisition of automated non-destructive testing installations with real time 3D display (a first in 1995)
my web application framework: Automatic generation of really complex web applications (the whole stuff from the HTML/javascript front-end to the server back-end). It has been fully Ajax-based with dynamic modification of the displayed pages without reloading the page. This seems obvious nowadays but it was also a first in 2000 and IIRC the term "Ajax" was only invented 5 years later.
hpcc: The awesome DSL to VHDL compiler. In many aspects VHDL is the complete opposite of Common Lisp. It's a hardware description language used to program FPGA and it deals with the lowest possible programming level with data types like signals, clocks, and bits. Programming at that level is really tedious, time consuming and verbose so it's really a relief to be able to generate tens of thousands of lines of highly optimized VHDL code from just a few hundred lines of some high level Domain Specific Language.
cl-pdf/cl-typesetting: Being able to generate the first PDF files from scratch in only 107 lines of Common Lisp was an Haha moment.
What you dislike the most about Lisp?
The language itself is somewhat good enough and anyway Common Lisp makes it really easy to change most of itself to add the new and cool stuff or ideas of the day.
In fact what I dislike about Lisp is outside the language and more related to the (mis)perception that people have about it. Having almost everytime to justify its use and sometimes even to fight to be able to use it is somewhat annoying and tiresome.
Of course, Lisp is not for everybody but this is not a reason to have nobody using it. In fact I view Lisp as some kind of amplifier which will give awesome things when used by brilliant developers but will end up giving an incredible mess when used by people without any clues about what they are doing. That's one aspect that makes Lisp very different from other languages which are especially designed to try to normalize and average what people can do with them.
Describe your workflow, give some productivity tips to fellow programmers.
I obviously use emacs and all the nice stuff that work with it like slime, org-mode, magit and lots of other packages. I switched from subversion (svn) to git for all my projects and I now use git mostly form emacs with magit.
In general I have several instances of Lispworks running as standalone apps on my laptop but connected to Slime through M-x slime-connect rather than being started by slime. I do a lot of exploratory programming and interactively play with the code while I write it and if I have to optimize some code I very often use #'disassemble.
About my Lisp coding style I really do like generic functions, CLOS and the MOP. As mentioned earlier I try to generate as much of the code as possible by using macros in simple cases and full blown DSLs in more complex ones. Sometimes I do not dislike making some premature optimizations when I know that speed will be important for some project. BTW when speed matters I like to generate and compile code on the fly when it's useful (and possible) and try to generate really optimized code. It's always cool when you have Common Lisp applications running much faster than C++ ones.
I also refactor a lot. Generally when I start on some new stuff I try to make a first version that works well enough but then once it's done or generally after some time I have some cool ideas to make something much better. At that point I'm really happy that Common Lisp makes refactoring very easy thanks to optional and key args, macros and generic functions with multiple dispatch because all that enables me to make even very deep modifications very easily. I really find those Common Lisp features make software very resilient to code modifications.
What are the advantages of using Lisp in the HPC field? What are the drawbacks? Are you happy with your technology choice?
The HPC field is huge and I can only talk about the small corner of it I know which is the ultra-low latency processing of vast amounts of data we do in NovaSparks. For that the various DSL compilers generating VHDL code have really been a key enabling factor. Programming FPGAs is notoriously difficult and time consuming and this is the major factor limiting the use of reconfigurable computing in the HPC field. The use of those DSL compilers has made it possible to use those FPGA on a more practical basis for processing financial data.
BTW I'm looking for other HPC domains in which those DSL VHDL compilers could be used so feel free to contact me if you have some ideas.
If you had all the time in the world for a Lisp project, what would it be?
Again that's a difficult choice. Speaking about pure Lisp projects I've been willing for a long time to clean up and modernize my web app framework before releasing it as open-source so maybe that could be a good project to start.
Otherwise as mentioned above, I'm looking for possible applications and opportunities of leveraging the DSL => hardware compilers and the FPGAs around some Big Data processing. I have a few ideas but nothing specific for now.
Anything else I forgot to ask?
A conclusion? (Everybody wants a conclusion.) So here is mine:
Common Lisp is Awesome! It is much easier nowadays to use it thanks to all the projects and libraries that Quicklisp makes available. If you do not know Common Lisp then learn it and this will make you a better programmer anyway even if you do not use it directly after.
David recently pulled in a couple of changes to make OB-SML happy with Emacs’ package manager, SML-MODE 6.4, and Org Babel. For you reproducible research junkies, this is super-cool. It is already out on Marmalade.
I found another one (I guess I still need to order that book). In using the UVM, I have some sequences randomizing other sub sequences. I really want it to work like this (simplified, non-UVM) example:
class Foo;
randint bar;
functionvoiddisplay();
$display("bar: %0d", bar);
endfunctionendclassclass Bar;
int bar;
functionvoidbody();
Foo foo = new();
bar = 3;
foo.display();
assert(foo.randomize() with {bar == this.bar;});
foo.display();
endfunctionendclassmodule top;
Bar bar;
initialbegin
bar = new();
bar.body();
endendmodule
See the problem there? Here's what prints out when you run the above:
bar: 0
bar: -1647275392
foo.bar is not constrained to be 3 like you might expect. That's because this.bar refers to bar that is a member of class Foo, not bar that's a member of class Bar. As far as I can tell, there is no way to refer to bar that is a member of Bar in the constraint. I guess Foo could have a reference back up to Bar, but that's really awkward. Has anyone else run into this? How do you deal with it?
Shane Celis has announced that Emacsy was accepted as a Google Summer of Code project. Celis describes Emacsy as an embeddable Emacs-like library for non-text applications: Emacs OS without the text editor. The idea is to bring the Emacs way of doing things to other applications. Emacsy will be built on top of Guile Scheme, which is already embeddable in C and C++ applications.
Celis’s post gives a hint of what he’s aiming at but for the real details you should see his Google SoC application. It seems like an interesting project and I’m looking forward to seeing the results. Take a look and see if you don’t agree.
Some years ago, I came across M-x animate-birthday-present (and therefore animate-string and animate-sequence) while reading through the output of M-x apropos-command RET . RET, which lists all of the interactive commands. (Well worth exploring! The Emacs Manual also lists a few unusual things under “Amusements.”) It’s one of my favourite examples of odd things you can find in Emacs, like M-x doctor and M-x tetris. I use animate-string to create the title sequences of Emacs chats like this one with Bastien Guerry.
It turns out that lots of people use the Emacs text editor for animating things. Andrea Rossetti (from Trieste, Italy) e-mailed to share this little thing he put together to simulate typing in Emacs. And, boggle of boggles, someone even taught a course on Emacs Lisp Animations.
Next: Maybe someone can make an onion-skin animation mode to go on top of artist-mode so that we can make Emacs flipbooks?
Today’s exercise comes from our unending list of interview questions:
Find the longest run of consecutive characters in a string that contains only two unique characters; if there is a tie, return the rightmost. For instance, given input string abcabcabcbcbc, the longest run of two characters is the 6-character run of bcbcbc that ends the string.
Your task is to write the requested program. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.
The European Lisp Symposium (ELS) is over now, and it’s been pleasant experience: thoughtful discussions, beautiful city, and parentheses all around. Thanks to all the Lispers and Schemers who made it to ELS for the friendly atmosphere!
The slides of the talk I gave on the design and implementation of Guix are available on-line. Nick Levine also published audio recordings of most of the talks (thanks!).
You can now read Wondermark — and dozens and dozens of other webcomics — in an iPhone app, thanks to Comic Chameleon!
Comic Chameleon’s the brainchild of my friend Bernie Hou, a comic creator himself with “Alien Loves Predator”. There have been lots of other apps out there that harvest webcomics without permission, usually by scraping the images off our servers, and the app creators often charge for the app without paying anything back to the creators. Bernie’s doing it right — he contacted me and a bunch of other cartoonists, offered his development skills, worked out permissions and contracts, and I was thrilled to be a part of it. Now the app is out, and it’s great!
You can read, browse, and share the entire archive of Wondermark (and many other comics) totally for free — the app is ad-supported. (Bernie’s looking into a way to charge a buck or so to remove the ads.) An Android version is also in the works. But especially for a version 1 product, Comic Chameleon is really great and I recommend you download it and try it out! More information at the official Comic Chameleon site, natch.
“Hooray, now I can read the mouseover text on my mobile device!” — psssstt, click the black word balloon icon to the right and below each comic. The magic was inside you all along!
The one downside (in my opinion) is that Comic Chameleon only displays the comics, not these scintillating blog posts. So, enjoy reading comics on your phone or iPad — just remember to check in periodically to see what’s going on down here! Or, the official Wondermark Facebook and Twitter feeds will ping you whenever there’s something cool to check out, too.
Thanks, Bernie, for building such a neat app, and thanks to all of you in advance for giving it a whirl!
Previous 
