Category: Clojure

funcjure

funcjureI’ll soon be starting a new permanent job, and in preparation I’m making it clear what is my prior intellectual property. In this case, it’s not so I can make “oodles of money” in the future, but it’s actually so I can release it as Open Source.

So what I’ll be explaining here is not something I’ve finished, but an idea which I’ve actually been playing around with for a year or so, but never gotten around to fully implementing. This will also help bring together my thinking on this and hopefully inspire me to really get going on this project :-)

 

What is funcjure?

funcjure is a “functional syntax wrapper” around Clojure. What do I mean by that? Well, Clojure is a great language, especially if you’re either used to or prepared to adapt to prefix notation. ie instead of typing 1+1, you type (+ 1 1) which although relatively easy to understand, can get a bit harder as things get more complex, like 9+5*7/4+62/6 which would translate as (+ 9 (*5 7) (/ 62 6) etc…

The logical question is Why? (do I have to type prefix notation) The Clojure and Lisp people will say “well, that’s the way it is, so just get used to it”, which I’m fine with as I wrote my first Lisp program over 3 decades go. Even still, I would prefer to use infix notation (which is what we’re taught for maths) and see no reason why we shouldn’t as computers are great at doing the sort of rote translation required to convert infix to prefix notations.

Clojure is a great language, which also has a fantastic ecosystem and community, and is written in Java which has a HUGE ecosystem. Furthermore, Clojure can call Java code, which has helped tremendously by giving Clojure “out of the box” access to so many libraries and products. Finally, because Clojure is a Lisp, its strong point is Symbolic Manipulation which is exactly what is required for translating infix to prefix structures in order to implement funcjure!

What would this look like? Let’s take some typical Clojure and then show what it would look like in funcjure:

"Hello World!"            ; Minimal "Hello World!"
; "Hello World!"

(println "Hello World!" ) ; Standard "Hello World!"
; println("Hello World!)
; Clojure((println "Hello World")) ; Execute some Clojure code
(def a "test")            ; Define a variable
; a="test"

(def mylist '(1 2 4 5 6)) ; Define a variable list
; mylist='(1 2 3 4 5 6)

(println a mylist)        ; Print our variables
; println(a mylist)

(first (rest '(1 2 3 )))  ; Get the 2nd element of the list
; first(rest('(1 2 3)))

(.println System/out "Hello World from Java!")
; System.out.println("Hello World from Java!")
; or Java(System.out.println("Hello World from Java!"))

(defn factorial           ; Now let's do the classic Factorial function
  ([n]                    ; when only one argument is passed in
    (factorial n 1))
  ([n acc]                ; when two arguments are passed in, with recursion
    (if  (= n 0)  acc
    (recur (dec n) (* acc n)))))
(factorial 6)             ; And test it
; factorial(n) = factorial(n 1)
; factorial(n, acc) = if((n==0), acc, recur(dec(n), n*acc))
; factorial(6)

The overall purpose is to make code much more accessible and ‘natural’ to write as we’re essentially taught infix notation as our “second language” when we study even the most basic mathematics. None of this is really new – in the beginning, things look like our old friend BASIC, with direct assignment and loose typing, which easily map to Clojure. You’ll also notice that access to Clojure and Java is provided by the respective functions. Speaking of functions, these are written similar to Prolog and other languages (eg Erlang) which allow for “pattern matching” in function definitions. In the beginning, the capabilities would be mapped directly to Clojure (as shown above), but eventually it would be nice to go to a full Prolog style, so the factorial function could be written like

factorial(0) = 1                ; The base termination case
factorial(n) = n*factorial(n-1) ; Iteration by recursion

which I think is way more elegant.

That’s about it for this post, as this really contains the base concepts for what I’d like to do. There’s obviously a lot more scope and subtlety to this, some of which I’m aware of and probably a lot more which I’m not, but I personally would much prefer to use a language like this and have access to Clojure and Java when needed for efficiency or easy code porting.

What do you think? All comments / suggestions / critiques would be gratefully accepted as I haven’t really done much more work than this, other than having a 1/2 working translator for this syntax which I’m going to (hopefully) get to work on in the next few weeks before I start my job.

Back in Australia

MoranReserve

WOW! I just realised that it’s been a year since I did a blog post. I knew it had been a while, but I figured around 6 months max… Well, I suppose I needed it, and the good news is that I’M BACK!

I’m not going to go in to what has transpired over the past year since an agile adoption, as it’s quite involved and will come out in some successive posts, but will focus on where I am now and what my Posting Plans are.

Near the end of last year we started planning a return to Australia after almost a decade in the UK – what a BLAST! So many lovely experiences, people, places and new friends :-) Eventually though, it was time to return to Australia as it really is my home and although I may have a few gripes about it, Australia really is an amazing place and as the old cliche goes: I now really appreciate how lucky I am to live in this country after being away so long. That shot above is from my local beach which is 3 mins drive, and I can get in to the city (in the distance) in around 1/2h with a walk and public transport – that’s a pretty good lifestyle.

So what’s in the pipe? A much more varied mix than when I started out doing just “software stuff”:

  • Reflections on life in the UK and Europe
  • Observations on life in Australia now I’m back
  • Reviews of gadgets
  • Travelogues as I explore my own country – now I’ve probably seen more of Europe than I have of Australia, so it’s time to correct that
  • Process  – yes, I’m still on the agile path
  • Architecture – looks like that will be my primary area of work still
  • Software – which I’m gradually getting back in to, with the current focus being Clojure
  • Probably the odd bit of music or photography to round things out
  • Anything else anyone would like me to write about…

YachClub

Welcome 2015!

London Fireworks 2015M25 Carpark

It’s been quite a while since I’ve posted much, mainly due to a contract which requires me to drive on the M20 and M25 (aka “the carpark” for those outside the UK) and as a result, I just don’t seem to of had the time and energy…

Javelin TrainOh, how I long for those lovely trains, and will never complain about a 30 or even 60 minute delay – the worst I’ve had in a car is a 1h trip taking 4h!!!

I look forward to getting on something where someone else is doing the driving so I can use my time effectively

Amazingly, it seems like only 7% (4.5 Million) people in the UK use public transport. Given that nearly 1/3 (22 Million) live in the South-East, where transport is generally pretty good, that seems pretty low. No surprise given the number of people on the motorways – I’ll be happy to take one more off them next contract.

So what’s up for 2015 for me and this blog?

For one, I plan to start getting back in to a bit more of a rhythm, both with my posts and the associated (play) work (generally outside “real work”), and I will continue to post based on my experiences – recent and past…

theme : ecologyWhat are the themes though? Here’s a list of where I’d like to go:

  • Lifestyle & Reviews
  • Process & People & ScramJet
  • Clojure
  • Architecture, including Enterprise & SOA aspects

in no particular order. I won’t get in to specifics as much of it is not yet planned, or I’m working on it but don’t want to reveal it until I have enough meat on the the bones so I can be sure it will fly.

teaserHere’s a few teasers though based on posts I know I’ll write or have in draft form:

  • Review of Bob Marshall’s “Thinking Different” happening last year
  • Review of the: BMW i3 electric car; Samsung Galaxy Alpha
  • Corporate Subversion – in a positive manner of course :-)

and that’s really just the “boring stuff” – there should be some very interesting posts coming as I hit my stride.

I hope you’ve all had a great XMas & New Year break and look forward to some great interactions in 2015!

Changearc – the thinking person’s RiczWest ;-)

Today I’m (@RiczWest) starting a new twitter account. It’s called ChangeArc. For those who follow me, you’ll recognise this as my “Blog Name” – so what’s the purpose?

Simple – when I started using Twitter, it was primarily as a bookmarking tool. To some extent I still use it as such, but also for so much more…

There’s one problem though – as I look at a lot of content, that means a lot of tweets, and not everyone likes that, including me! There are a number of great people I’d like to follow, but I can’t because they tweet too much.

To that end, I’m going to start a much lower volume (only a few tweets per day) account which is ChangeArc. So what can you expect apart from less tweets? Extremely high quality tweets that will include any posts I do.

I don’t know how this will evolve, but it will be interesting to see…

EuroClojre – Day 2

Wow! I thought Day 1 was Great – Day 2 just as good, including some great but “enterprise geeky” topics like Enterprise Integration in Clojure and A Perfect Storm for Legacy Migration ;-) Plus, we had Stuart Halloway’s…

Keynote – Narcissistic Design

20131015-104648.jpg

  • Stuart Halloway
  • Examples
    • Build Tools
    • ORM
  • Advanced
    • Integrating narcissism and agile practice
    • Technologies to avoid
      • Functional languages & databases
    • Never eliminate complexity, automate around it
    • Manage by pull request, because code is the first and best unit of discussion
  • Top 10
    1. Use OO, and don’t forget those setter methods!
    2. Prefer APIs over data
    3. Start with DSLs
      – Nothing says “screw you” like a DSL
      – JVM
    4. Always connect (and never enqueue)
    5. Create abstractions for information
      – Encapsulate!
    6. Use static typing across subsystem boundaries
      – Exceptions
    7. Put language semantics on the wire
      – Focus on programming rather than data languages
    8. Write lots of unit tests
      – Example Based Testing
      – TDD
      – Always be coding
      – Never get on the hammock
    9. Update information in place
      – The one thing about OO: gives you all the complexity but no history
    10. Leverage context

Using Clojure to Serve The Internet of Things

20131015-110116.jpg

  • Kontiki
    • Lightweight OS
  • Nest
  • How do I get iot on my CV?
    • Dev kits
      • mBed
      • Electric imp
      • RasberryPi
      • Arduino
    • Beagle bone
    • Open data sets
    • Web / Mob apps
  • Xively
    • LogMeIn
    • Went from Ruby
      • To Clojure
        • Order of magnitude improvement!
  • Architecture of “thermometer”
    • Twitter
      • Clojure hero ku
        • Clojure Xivi
          • ?
  • IoT
    • 1.1b PCs
    • 2mm2 32-bit computer with wiFi for $1
      • 32kB RAM
      • 128kB Flash
      • Radio
      • Weightless – OS?
        • Batt
          • 50 bytes/h for 10y on 2 AA Batt!
    • 5.8b mob
    • 1T Things
    • Main challenges
      • Discovery
      • Directories
      • Provisioning
      • Security
    • Should take off due to economies of scale
    • Massive Data Volumes
      • Motor
        • 1Gb data / h

Newspaper Massacre!

20131015-123343.jpg

  • Conways Law
  • Daily Mail
    • Mail Online
      • Largest online newspaper
        • Serve 154Bn images!
        • Cached by Akamai
        • Serve 1000 articles/s
      • Very long home page
  • Mobile
    • Gamechanger
      • Personalisation
        • Trying to scale
  • Published messages from Legacy to Topic
  • Challenge
    • Render the homepage
      • No caching
      • Response <300ms
  • Mustache
    • Templates Library
    • Themes
      • e.g. Mobile
  • Libs / Projects
    • Memcached
    • Mule
    • Reddis
    • Riemann
      • Graphite
      • Logstash
    • GitHub.com/mailonline
  • Ppl
    • @clifcunn
    • @jonpither
      • Pitheringabout
  • Config
    • Avout
      • Wrapper around Zookeeper
  • Services
  • Built a library around software infrastructure
  • Future
    • Rewrite everything
      • ~13kLOC
  • Devs sit next to newsroom
    • E2E Ownership
    • Responsibility
  • Terrible Legacy
    • Using Spring

(into reduce transient)

20131015-125828.jpg

  • Clojure.core/ancestors
    • (Ancestors (class []))
  • Libraries
    • avl.clj
    • core.rrb-vector
  • Guy Steel Talk
    • About sequential reduction

Lightning talks: Literate Programming, Bare Metal Lisp & Templating

20131015-153632.jpg

In search of workflow nirvana: Clojure, Emacs, Org

    • Rich Hickey
      • Ant Simulation
    • Org – mode in EMACS
    • Literate Programming
      • “is what you need to rise above the ordinary level of achievement”
    • Pedestal
    • Working on functional CMS system
    • hackerretreat.com

Lithium: a small Clojure-inspired Lisp on the bare metal

      • Compiles to x86
        • No runtime dependencies
      • Fluchtpunkt Lisp
      • Paper: An incremental approach to compiler construction
      • Movitz
        • Similar for Common Lisp
      • Synthesis: an efficient implementation of fundamental operating system services

Templating In Clojure

    • Hiccup
      • HTML as a data structure
      • No real separation
      • Good if full stack devs
    • clj-jade
      • Successor to HAML
      • Use Jade for J
      • Or moustache
    • Enlive
      • Transformations of HTML
      • Good for front-end
      • CSS Selector based
        • Can break transformations
      • Lacks HTML abstraction
        • + Hamilito

Enterprise Integration in Clojure

20131015-173247.jpg

  • clumsyjedi @ GitHub
  • Enterprise Integration Patterns
  • Looked At
    • Mule
      • Not used – too many problems
    • Camel
      • HolyGrail
    • Use
      • core.async
      • + Threaded Macros
    • Lamina
      • Event Stream Processing Library
        • Superset of EIP
      • Rysome
    • Storm
      • Java / Clojure Hybrid
        • Chosen
    • ClamQ
      • Clojure i/f for ActiveMQ
    • EEP
      • Good visualisation
  • Areas of Interest
    • Systems Integration
    • Business Rules Capture
    • Distributed Systems Orchestration
  • Using Vim
  • Beyond Patterns

A Perfect Storm for Legacy Migration

20131015-181427.jpg

  • Twitter Storm
  • Clojure
  • Distributed runtime
    • For “eXtreme Processing”
  • Apache Thrift
  • Nimbus
  • ?
  • Gives
    • Stream Processing
    • Realtime Analytics
    • Continuous Computation
    • Distributed RPC
  • Concepts
    • Bolt
    • Spout
    • Topology
    • Grouping “field”

EuroClojure – Day 1

I’ll be putting some notes / mind maps up here from EuroClojure over the next two days. In the “agile spirit” I’m including my notes fairly raw below and may refine them later… For the meantime, in case you weren’t able to attend EuroClojure, here is a summary of some of the fun you’ve missed ;-)

Keynote

EuroClojure Keynote MindMap

Critical Theory

  • Nomad
  • Rhizome
    • “Interconnectedness of Things”
  • State
  • Information
    • Collapse of Abstracton to Reality
      • Brazilia

Software

  • Production methods
    • Prussian “Scientific Forestry”
      • Create Flawless Abstractions
    • vs Organic Forests
  • ~ Forest

Greeks

  • Techne
    • Timeless
  • Metis
    • Cunning
      • Local
      • Contextual
      • Nomadic

Refs

  • Patterns of Software
  • Fictions and the aleph
  • Data and reality
  • Seeing like a state
  • Simila and simulation
  • Invisible cities

Alexander

  • The Quality without a name
    • Contextual
  • Richard Gabriel
  • Pattern Language
    • To express patterns that were observed (locally)

Gerald Weinberg

  • An Introduction to Systems Thinking
  • Heuristic Devices don’t tell you when to stop

Make your own Lisp

Make your own LispBODOL

  • Homoiconic
  • Immutable
  • Typed
    • Robin Milner
    • Example
  • Pure
  • Makes HN Caremad
  • Pattern Matchinng
    • Used core.logic

Instaparse

  • Mark Engelberg
  • EBNF
    • Or Parse Combinators

Refs

  • GitHub.com/bodil/BODOL
  • GitHub.com/bodil/building-lisp

Misc

  • Core Match
    • Not used as specific to Clojure…?
  • Lispkell
    • Also Lisp + Haskell

Life in a Browser

Life in a Browser

Genetic Algorithms

  • Breed s-expressions

Zipper

  • Tree handling Library

Resources

  • Complexity – a guided tour
  • Complexity explorer
  • Stephen wolfram
    • A new kind of science

Biologically inspired computing – Artificial Life Project

  • In browser with core.async
  • Enlil
    • Head Mesopotamian god

20131014-154848.jpg

Liberator

Liberator

Free your data with RFC2616

  • Http 1.0
    • + Patch
      • Liberator doesn’t handle
    • + WebDav
    • Almost 40 decision points for all status codes

Only 1 use in prod – Clojure library

  • Implement REST Resources
    • On the server side

~ 1/3 LOC Takes care of most of the status code processing

  • Uses “standard” from http flow diagram
    • See web site

Automatically converts your data to JSON or EDN

Creative Machines

  • Creative Machines

    Book: Creative Music

    • Death = Entropy Chaos*
  • Recombinance
    • Creativity does not occur in a vacuum

    Influence

    • Serendipity

    Learning + Inference + Knowledge

    • Canon Builder
    • Association Network
      • Like Neural

    New neural networks

    Cognitive Bias

    Examples

    • Death Dance
    • Aaron
      • Painter
      • Howard Kern / Cohen
    • Music (overtone)
      • Markov Chains
      • Neural Nets

    David Cope

    • Emily Howell
      • Music

    Creativity

    • Can happen by
      • Combination
      • Exploration
      • Transformation
        • Reflect on System Rules
        • And Change those Rules
    • Psychological
    • Historical

    Functional

  • Functional 3D Game DesignNeed to get everything done in 16ms
    Channels

    • Side effectual
      • => !used

    Events
    Callbacks
    World
    Based on JMonkeyEngine
    Probably go to LWJGL
    Use Quarternions
    Vector

    • 3 imaginary numbers
    • 1 real number

    Functional Reactive Programming
    Modelling both Continuous and Discrete Data changes

Common Clojure Smells

Common Clojure Smells

Martin Fowler & Kent Beck

  • Refactoring

Long parameter lists

  • Underlying issues

Clojure specific smells

  • Locally scoped atoms
  • Magic Keys
    • aka Data structure coupling
    • Hard to figure out the cost of change of key names, data types etc…
    • Keep data aware fns together (defrecord?)
  • Indirection by partiality
  • Macromania
  • Parents Proliferation
    • aka Over-nesting
    • Trying to do everything in one expression
    • Balance between tenseness & readability
  • Lazi-itis
    • Excessive use of lazy evaluation
    • Many discrete steps with a final one that executes the seq
    • Consider pulling things in to coarser-grained steps to avoid long pipelines

Kent Beck

  • Smalltalk practice

Jennifer Smith

Do OO Heuristics apply to Functional code?

Is Java the new C?

JavaLogoI started programming in Java a few years after it was released. I had been doing C++ for a number of years and just come off a C++ project which failed because they (who were a pretty good team of programmers) couldn’t scale it because of the two common killers: Threads and Memory Management. Near the end I was reading James Gosling’s original paper on Java which addresses the major problems of C++ and explaining how Java addressed them. I was obviously quickly sold! As a result I spent over a decade programming in Java and J(2)EE, in the beginning implementing (mostly web) projects way faster than the “C++ laggards” could.

Now, I’ve been doing Solution and SOA Architecture, occasionally whipping up a prototype in Java, or doing some Perl but quite often working with WSDL, XSD’s and just plain models. Recently though, I’ve been playing around with Clojure in my own time and am amazed at how much fun it is. It really reminds me about my early experiences with Java, but that would be the subject of another blog post…

Recently it has struck me just how many languages have versions or been constructed to run on the Java JVM. A list from Wikipedia shows:

Java LanguagesThat’s quite a few languages! What’s more, a number of them have plugins to an IDEs such as IntelliJ or more commonly eclipse. It struck me that these languages were effectively using Java and often IDE’s as a runtime / development platform, much like many high level languages used C.

Which begs the question – where is Java headed in the long-term? Yes, they’re adding Lamda’s in Java 8, but why would I want that when I can work with Clojure which has had this from the beginning and has a whole bunch of good extra stuff… Is it destined to become a “low level language and environment” upon which more sophisticated languages will be built?