[Aldor-l] Literate programming

[Thomas J. Moore]

On 08/19/08 17:06, Ralf Hemmecke wrote:
> Welcome. Or have you been listening already for quite some time?

I first discovered Aldor in late '05, and subscribed to the list shortly 
after.  Unfortunately, I also started a new job (in a new city) at the 
same time, and have pretty much abandoned all personal projects since, 
as is my habit.  So, I just lurk.  And then post long brain dumps.  Sorry.

> My first question. Why are *you* interested in Aldor?

I liked Ada for its features, but not its verbosity.  C++ has issues as 
well.  I had planned to just make my own hybrid as an extension for my 
never-finished Ada compiler, but have also always kept an eye out for 
languages which already do what I want, because I'm fundamentally lazy. 
  I'm not sure how I stumbled onto aldor (I like CAS, but I thought I 
discovered aldor before axiom), but it has nearly everything I want: 
strong typing, better generics (type-parametered code) than Ada, terse 
syntax, type attributes (of a sort), return type overloading (unlike all 
the C-like languages), and minimal overhead.  Of course I really loved 
Ada for a long time after just reading (and even implementing my own 
interpretation of) the spec and doing simple tests, too, until I tried 
to implement a large project (my literate programming editor project) 
using gnat and found it more and more constraining, partly due to my 
overly generous misreading of some key portions of the spec.  I'm not 
yet experienced enough with aldor to say I'll continue liking it 
forever.  I wrote a long-winded Ada-vs-C++-vs-C-vs-what_I_want document 
that I thought I had written an aldor equivalent for, but I can't find 
it right now.  In summary:  I don't really care about the mathematical 
underpinnings; I just think aldor may be a very good general purpose 
programming language.

> Oh. That is simply not true. I agree that mmm-mode sometimes get's 
> confused with dollar signs, but as the mmm-mode maintainer told me that 
> is a problem with (x)emacs. That editor is simply not made for switching 
> major modes in the middle of a file.

Well, OK, perhaps I should have said "in my experience" it gets confused 
far too easily in both emacs and xemacs.  It looks and works OK at 
first, but as I go to later chunks, it gets worse and worse, and gets 
intolerably slow to boot.  Adding and removing chunks almost never works 
right, and multi-language support is also impractical at best (magic 
comments at the start of every chunk???).  Maybe it's just my coding 
style, or maybe it's the languages I used during testing (and no, I 
almost never used dollar signs in either C or Ada, except maybe in a few 
limited literal strings).  In any case, I spent many days (or was it 
weeks?) trying to make mmm-mode or noweb-mode work right.  I still put a 
noweb-mode magic comment on my files in case I ever feel like going back 
(and because I copy & paste a lot), but mostly I've abandoned emacs. 
Then again, it might do a better job than vim, but I've never tried vi, 
nor even wanted to:  I hate vi and all its clones, even though they are 
much better than they used to be (unless you have to use it, which is 
usually because you're on commercial UNIX where all the command-line 
utilities seem dated or otherwise broken), no offense intended.

> If I spot some place that I further want to work on, I simply click into 
> the dvi viewer and voila, emacs suddenly opens the corresponding file at 
> the correct place.
> That works because of "inverse search".
> http://www.hemmecke.de/aldor/allprose/myalpssu21.html#x62-5800024.3.9

Yes, I think inverse search is a neat feature, and I was aware your 
system supported it.  It might be just what Pippijn was looking for, but 
  it's definitely not for me.  It doesn't fit my programming style, and 
I just don't see a problem with using a plain search.  I also abandoned 
DVI as an output format for my own build system because it's 
non-standard, and xdvi crashes a lot for me in the last year or so. 
xpdf works fine, though, and I can attach my noweb sources to PDF for 
easy distribution (I could probably try to figure out how to make a 
\special to attach to DVI as well, but I'm not going to bother).

>>> This requires me to look for the string "Do something interesting here"
>>> in the nw document. That takes me time and I quickly get annoyed when
>>> things take time.
> 
>> The noweb-mode on emacs does support a menu and side-tree and 
>> hypertext-like commands (I think), but here I use PDF as well.
> 
> Like me with the DVI file. But Pippijn is right. The editor should 
> simply be intelligent enough to unfold the code inside the editor and 
> even letting me edit the (remote) chunk here. If only there were a smart 
> elisp programmer who could do that... sigh.

And of course elisp is the biggest reason I avoid emacs.  I have an 
unnatural aversion to lisp and any similar language (surprisingly, this 
does not taint my view of aldor).  I tried to give emacs and xemacs a 
fighting chance when I wanted to just use something someone else wrote 
instead of writing my own editor, but both failed miserably without 
requiring massive time and probably coding on my part, which I just 
don't want to do for an editor I don't like that much anyway.  As to the 
unfolding feature, I guess leo might do that sort of thing (which you 
actually mention yourself, so you must have looked at it at one point). 
  Again, I never felt it necessary, because it doesn't feel like too 
much typing to just ctrl-spc ctrl-e esc-w ctrl-\ ctrl-y enter (mark, 
eol, copy, search, paste, at least in my current key bindings).  As long 
as the editor is fast (and emacs isn't, really, which is another reason 
I don't like it), searching is mostly painless.  I guess it's what 
you're used to.  It's not like searching normal code is any easier, 
really, unless you write really small functions and take advantage of 
the menu and ctags.

>> I have invested a lot into making my noweb output work the way I like.
> 
> I've glanced over your package (http://storage.iu.edu/qmgr/build.pdf). 
> You seem to be quite knowledgeable in perl, shell, makefile, latex, etc.

Well, shell, sed, and make, but not really perl or latex.  I use what 
little I know plus the crutch of manuals to get what I want out of the 
tools I have.  I actually have an unnatural aversion to perl as well, 
but sometimes you have to use what's available.  In particular, I use 
perl because shell, awk, and/or sed were too slow or just not capable of 
what I needed, and I didn't want to require compilation before building 
(at least not in v3; I did use C in v1).  I actually like latex, and can 
write complicated tex code, but I probably do a whole lot of basic 
things the "wrong" way due to lack of review by people who know better. 
  I used to use lyx to compensate for my lack of basic latex knowledge, 
but lyx is definitely not even remotely a programmer's editor.

> I have the impression that I could learn quite a bit from your work. 
> Whenever I will revise ALLPROSE to move it to use autoconf/automake, I 
> must certainly contact you for a lot of stuff that I could improve.

I'm willing to help.  I was planning on writing my own aldor extension 
to my build system as soon as possible, but I don't have enough 
experience yet to know exactly what I want.  At least I want a decent 
code formatter.  A simple syntax highlighter may be enough (like a spec 
file for highlight, listings, and/or GNU source-highlight), but maybe 
something that can convert aldor to something even prettier a la (c)web 
would be even better.  It would be helpful in that respect to have 
something like the Ada Style Guide for aldor.  I'm not even sure what 
"good" aldor should look like, and I am probably just imposing my C 
habits on it.  I can look at the standard libraries' source code for 
consistent trends now that it's available, but I haven't had a chance to 
do so.

> I would quite be 
> interested in your experience with the listings package.

Well, the main issue I have with listings is that it tries to compensate 
for different font sizes, but does not do a very good job of it.  The 
HTML output in particular ends up mangled sometimes, although that's 
partially tex4ht's fault (it doesn't work with l2h at all).  Poor HTML 
output is the reason I prefer highlight for my own stuff right now. 
Other than that, it's like any other dumb syntax highlighter:  a few 
specialized rules for lexical analysis; hope that your language doesn't 
violate one of its built-in assumptions (I think aldor should work).  My 
build.pdf/nw gives a few other caveats.  If you're lucky, your code will 
look pretty decent.  I am currently thinking of moving to GNU syntax 
highlight, as it uses a very precise state machine with regular 
expressions to do its parsing.  I haven't looked at how tolerable its 
output is yet, though.

>> I'm aware of
>> allprose, but I have not done anything with it, mainly because I use C
>> and Ada for nearly all of my code right now.
> 
> Right. ALLPROSE is focused on supporting the build of aldor libraries. 
> Pretty useless for anything else.

Well, not entirely.  Some features are code-specific, and some are 
documentation-specific.  Documentation-specific features are mostly 
universal.  That's why my build.nw is mostly about making documentation. 
    I don't doubt that ALLPROSE has features that would apply to any 
programming language.

>> I have yet to find the holy grail that allows me to
>> easily integrate source code documentation, API-level user
>> documentation, and end user documentation in a single document/source
>> file that also meets my need to have related things be as close to each
>> other as possible for maintenance and document flow.
> 
> Well, do you know of a mailing list where this could be discussed?

No.  There's comp.programming.literate, or whatever the group name is, 
but I haven't posted to usenet in 18 years, so I have no idea how (if 
it's possible at all any more).  The list itself seems dead as well, if 
Google is any indication.  I claim to be a "fan", but really I don't 
even know if I'm doing it right -- I want to combine two steps in the 
software engineering process that, in my experience, suffer if they 
become disjoint:  design documentation and implementation.  I don't want 
to write published literature, although shiny objects, er, nice 
printouts do appeal to me.  This is one reason I do/did not join any 
literate programming communities.  I use Norman Ramsey's tools, but I 
think I disagree with him on many aspects of its design, and am not 
looking forward to the next major version.

> My 
> experience with ALLPROSE tells me that exactly that holy grail is 
> missing. It is not so easy to satisfy all needs. I guess one must forget 
> about editing the actual plain text. Rather an editor is needed that can 
> switch to different views. The editor should arrange of small snippets 
> of code and documentation according to the view/goal one is writing for.

Yes, that was one thought I had.  Another was to use some sort of 
additional markup language that is easy to write and easy to convert to 
pleasant looking documentation in the appropriate sections.  I have 
spent much time pursuing the latter, but have yet to find something I 
like, or come up with an idea that I could implement easily myself.

The markup idea is part of why my build section allows you to at least 
duplicate documentation by marking document "chunks" with latex 
comments, and reinsert what would be extracted tangle-style into the 
document elsewhere.  build.nw actually allows you to include its "how to 
build" bit in your own documents using this technique.  I also use that 
e.g. to document functions I add to a scripting language where I define 
them, and then reinsert that documentation in the users' guide appendix. 
  I'd rather have the users' guide be a separate document, and perhaps a 
man page as well, though.  Part of the issue is that user-level 
documentation is traditionally man pages plus perhaps longer combined 
documents, plus maybe even a README, so really I want to combine 3-4 
documents into a single document without making it cluttered.  Maybe 
this seems insane to you, when I could (and probably should) just keep 
some of that stuff separate.

> I thought LEO http://webpages.charter.net/edreamleo/front.html was such 
> a tool, but it did not really convince me for what I have in mind. For 
> me LEO feels to code or structure concentrated. It does not feel as if 
> the documentation was written for humans.

Yes, I agree.  You can coerce it into using noweb for unstructured 
chunks, but then why use leo at all, instead of just writing plain text 
noweb?  My documents are structured (sections, etc.), but not *that* 
structured.  I also want a document I can print, not just browse, and I 
don't want the printed document to hide anything or be (much) less 
usable than the electronic version.

>> One of my wild dreams is to make an aldor-like compiler, written in
>> literate aldor (a change from my old dream, which was to the same in
>> Ada, but aldor does most of what I like about Ada, and is less wordy).
> 
> Oh that sounds interesting. An aldor compiler in literate aldor... that 
> is certainly a hard task for just one man. Why don't you start an open 
> project? Dictate your literate view and see what happens...

Well, if it wasn't hard, it wouldn't be worth doing :)  You may be right 
about opening up more, but either way I just need to spend time on it 
rather than other useless activities.  The problem with my ongoing 
compiler project is that I abandon it completely for long stretches of 
time (months/years).  I then spend a significant amount of time trying 
to remember where I left off, decide to reimplement major portions from 
scratch, and then abandon it again due to time or other constraints 
shortly after reaching full speed.  The same is true of most of my large 
projects, really.  I guess I don't have a very good attention span.

>> Missing bits are a nice formatter for aldor code I can incorporate into
>> my noweb build system, a more formal/complete language specification,
> 
> Well, that's true. The AUG is not really very formal. But then you could 
> write your own specification.

Yes, I actually started on this.  It's one of many pieces I still have 
sitting around waiting for me to get cracking on.  Since literate 
programming is (according to me) combining specification with code, this 
will also be a basis for the semantic parts of the compiler; I consider 
it the first step (I actually completed parts of the subsequent steps 
already as part of the Ada compiler project, but I'll probably end up 
rewriting those from scratch again, anyway).

  - Thomas J. Moore