My Thoughts on the Next Generation of IBM Domino App Dev

Before I started to work on Bluemix I had worked 15 years in the IBM Collaboration Solutions space, especially on application development. Below are my personal thoughts on how app dev in IBM Domino could evolve, both on-premises and in the cloud.

App Dev Models in Domino

Since Domino has been around for a long time the app dev capabilities have grown. At this point there are developers who build rich client applications via forms and views, there are developers who use Domino pass through HTML, XPages developers who use XPages for server and client side logic, developers who build REST APIs via XPages and frontends via client libraries, etc. The good thing for Domino developers is that none of the older capabilities were removed, but instead new functionality was added on top. I think it’s time to add yet another way for developers to build applications on Domino, again, without replacing anything that exists today.

The Power of JavaScript

While some Domino developers have started to use Java, my guess is that the majority of developers prefers scripting languages and I’ve heard many times the desire to develop all aspects of an application in one language. Fortunately with JavaScript you have this option today. In my little sample I used JavaScript on server side, for the web frontend, for the mobile apps and even the build script.

The popularity and success of JavaScript is amazing. There are 1 million tagged questions on StackOverflow, according to RedMonk there are more JavaScript projects on GitHub than projects in any other language and there are 250,000 npm packages available. I think it would be awesome if Domino developers could leverage this momentum and this community.

Node.js Runtime in Domino

Wouldn’t it be cool if Domino had a Node.js runtime so that developers could build Node applications that access data in NSFs via simple JavaScript APIs and JSON? This would allow Domino developers to leverage both the huge JavaScript community as well as the nice capabilities of NSFs like security, replication, documents and more. As done previously when adding XPages the core value of the Domino platform would still be used but supplemented with a new way to build application business logic and frontends. The difference would be that the tool Domino Designer wouldn’t have to be used anymore necessarily.

Collaborative Line of Business Applications

Based on my experience most applications built with Domino are collaborative Line of Business applications. Here is how I define those:

Line of Business requirements often lead to the development of new applications since standard software is not sufficient. Collaborative applications allow employees to work together synergistically to get their jobs done. Collaborative Line of Business applications often require the protection of intellectual property based on roles of employees. Samples scenarios are approval workflows, teamrooms or travel expenses.

Technically this often maps to requirements like authentication, authorization, rapid application development, REST APIs, web frontends, mobile apps, NoSQL databases and more. A Node.js runtime in Domino together with a way to authenticate against the Domino directory and a JavaScript APIs against NSFs including authorization could fulfill these requirements.

Adding cognitive Capabilities to Applications

While collaborative applications have been around for quite some time, I think the next generation of these types of applications will be significantly different. I expect new collaborative applications to heavily include cognitive capabilities, for example to provide highly personalized experiences, to leverage deep learning to detect important information for individuals via personal assistants or to use intelligent bots for conversations. Collaboration in the past seemed to have focused on the team aspects. In the future I think the personal aspects will get more focus and cognitive services and artificial intelligence can address this. Additionally natural user interfaces like speech recognition and gestures will become more and more adopted and context information from the Internet Of Things will be used to optimize collaborative experiences.

IBM provides this and more functionality, for example via services in IBM Bluemix. While Node.js applications are not the only way to consume this, they make it very easy and there are tons of examples for developers available today.

Rapid Application Development

With all the available tools, libraries and documentation I think it’s fair to call Node.js a rapid application development platform. However there is an extension of Node.js which makes it even easier to build collaborative Line of Business applications – LoopBack. LoopBack is an open source Node.js framework to build APIs and an extension of the popular Express server. Developers from the company StrongLoop are the main committers and IBM acquired StrongLoop a couple of months ago.

Over the last weeks I’ve tried LoopBack and I’m excited how easy it is to build applications. Business objects can be defined declaratively. The persistence, REST APIs, documentation of the REST APIs and client side JavaScript libraries are generated for you. Similarly to Domino ACLs and reader fields LoopBack supports role based application and business object access. Pretty powerful. Check out my sample project and the deck for additional information.

LoopBack comes with a connector framework to work against various databases. I think it would be great to add another connector for NSFs. Via Passport LoopBack also supports authentication against different directories and a Passport strategy for Domino could be developed similarly to the IBM Connections strategy.

Relation to Bluemix

The obvious question is how this relates to Bluemix and the cloud. It needs to be possible to deploy collaborative applications to Bluemix and use other Bluemix services, for example cognitive and IoT services, as described above.

Bluemix supports various runtimes like XPages and Node.js. So hosting the business logic on Bluemix is simple. The question is how to handle the data as long as the XPages NoSQL Database is not available for production usage. Here is an idea.

Bluemix provides several databases as services. One database I really like is the NoSQL database Cloudant which is based on Apache CouchDB. There are a lot of similarities to NSFs, certainly also because of the history. So why not using this database in the cloud? To make it easier for Domino developers to switch between on-premises development against NSFs and cloud development against Cloudant, maybe the Cloudant Node.js API could be implemented against NSF? Also if LoopBack is used changing the underlaying database is trivial and doesn’t require any code changes.

Closing Remarks

In addition to a JavaScript API (npm module) against NSFs there should be a JavaScript API against IBM Connections so that developers can integrate social functionality in their applications, for example social file sharing or status updates.

As I said these are just my thoughts and ideas. Just wanted to throw this out for discussion since people seemed to be interested in this at EngageUG and recent discussions I’ve had.

  • Patrick Kwinten

    A strength of NSF is that I can deploy it mostly anywhere in the data folder of the Domino server and it could/would work. How would you solve this with node and loopback? Must that be configured centrally or will I be able to set this up within my NSF?

    • Niklas Heidloff

      Good point, Patrick. I didn’t want to go in that level of detail and it certainly requires some evaluations. One option could be to use the StrongLoop Process Manager https://docs.strongloop.com/display/SLC/Using+Process+Manager. Other theoretical options could be to provide a Cloud Foundry runtime or Docker runtime in Domino. In any case I’d prefer a standard based way to package up applications rather than putting something more in the NSF.

  • Peter Presnell

    There is a lot of good thoughts there Niklas. JavaScript frameworks seem to be the way to go and cloud is getting a lot of attention. The challenge we have with our customers is how do we take the existing Notes applications and move them to the cloud and keep the lights on/enhance them at the same time as build new apps without the need to maintain multiple development platforms and/or expensive data migrations. The missing piece seems to be getting Bluemix to support Notes client access to the NSFs in addition to the REST API.

    • Niklas Heidloff

      Peter, to me it feels like if you want to move existing Domino apps in the cloud you should use a hosted Domino server, for example on Softlayer or offerings from partners, since my understanding is you’d have to change existing applications when moving the apps and their data to Bluemix. Again, this is my personal opinion only.

      With the XPages Bluemix runtime or the Node.js runtime in Bluemix however you can build new hybrid collaborative apps and access existing data from on-premises. There is a Secure Gateway service that can be used from Node.js applications on Bluemix and my understanding is that XPages has also an option to access NSFs on-prem.

      As I told Nathan I think it would also be great if your data integration/federation tool could host Domino data from on-prem in the cloud to build new or modernize user experiences.

  • Mark Roden

    I have been asking on the side for node.js modules to connect to Domino in the same way as we have mongo and cloudant connectors. To me that is a much more powerful story than XPages runtime in Bluemix. Data structure is the power, not the runtime.

    • Niklas Heidloff

      Sounds good, Mark.

  • John Jardin

    So, in short…a win win for everyone is for IBM Notes and Domino to include a Node.JS runtime and a couple of pre-built Node modules to integrate with Domino Data and provide Authentication?

    This means that Node.JS dev becomes an alternative to XPages development within the same domino environment?

    That would definitely spark some awesome capability…but it feels like much of what makes Domino Domino gets put one side, and only a small portion of the ecosystem gets used. This is the part that doesn’t make sense to me.

    • Niklas Heidloff

      Yes, to your first two questions.

      Not sure I understand “only a small portion of the ecosystem gets used”. But if my interpretation is right, yes, I suggest not to keeping adding new design elements and extending Domino Designer, but to leverage other very popular frameworks which require very similar skills that Domino developers have. Again, just my 2 cents.

    • Patrick Kwinten

      I would like to see graph capabilities in Domino so customers can remain their data in Notes and explore new ways to analyse the data they have. Perhaps more desirable than node.js personally.

      • John Jardin

        I don’t think Graph is far away from being a reality via the OpenNTF Domino API. Nathan’s using it aggressively. It’s a matter of getting it finalised and production ready.

    • Paul Withers

      Agreed on the last point. NoSQL = NSF but != Domino. Domino also offers mail routing and an API for workflow processing, a lot more than just CRUD. Agent Manager (for scheduled processes) has lagged behind, but the ability to enhance the front end of an application and leave data, integration to other applications and platforms, scheduled processes etc just working reduces the cost and speed of modernisation. For new applications, learning a new front-end is daunting, so being able to fall back to comfortable syntax for those areas *for now* makes an easier transition and also reduce cost / time, which keeps managers and users happier. (And I’m speaking from XPages experience here, where my first app was an intranet read only in XPages, with admin and editing done in Notes Client – my experience was much better, even on 8.5.0, than I see from newbies on StackOverflow.)

  • Rob Mason

    One observation that I have made at both EngageUG and at IconUK was the lack of “young” professionals there (I include myself in the “not young” category). Other tech conferences (especially ones focused on JS) are filled with younger professionals: these are the developers of the future; these are the people who need to be attracted to use Domino to maintain it as a platform. Likewise, any addition of current technologies will only help current Domino developers expand their skill-set.

    When Xpages was added to Domino, it made a big step forward in that it allowed us to use java in a way that it was being used elsewhere. Adding node.js to Domino would be a similarly huge step in the right direction. I don’t know a huge amount about how it works, but if it is added, I would like to see it added in such a way that it can be independent of the Domino version so that we avoid the big wait for java 8 that we have had already.

    • Niklas Heidloff

      Great points, Rob.

    • Patrick Kwinten

      Hi Rob, all those grey hairs made me feel comfortable at Engage, though I saw one guy trying to grow a beard and become a Strongloop hipster. Actually he joined us for dinner. His name is Martin :o)

  • Smicky

    IBM is already having enormous difficulties (certainly due to budget and ressources constraints) to maintain current technology stack and you want to add more in Domino core ? Hum, I’m sceptical.

    According to me one good strategy could be :

    – bring and keep the XPages stack up to date and don’t let it go outdated in tems of technical components : that means Java 8, updated OSGI container, updated Servlet container, updated IDE, move in the stateless direction (I know Phil R. had ideas on that).

    – for all the rest, Domino should be a “good citizen” in today’s architectures and technical landscape. Domino could be considered either as a repository or as an app platform. Considered as a repository, a NodeJS NSF Driver would help a lot (but it would require lot of effort to provide asynchronous driver to NSF). As an application platform, a REST API would help too. Note that in REST domain, WINK is also already outdated. In both cases (repository and platform), NSF would need a huge effort to stay on par we other nosql databases : expanded query capabilities, some transactional support, performance enhancements.

    – as of Bluemix, I see a lot of potential here but as long as NSF is not supported there, nothing will happen. (the % of Domino app not relying on NSF, at least for a very small portion such as conifiguration must be extremely low).

    • Niklas Heidloff

      Michael, since I’m no longer in the IBM Collaboration Solutions organization I can dream and don’t have to think about budget and resources :)

      But seriously, doing what I propose might actually be cheaper to achieve than either of your points “updated IDE” or “NSF in the cloud”. I suggest to leverage other popular and successful frameworks that IBM supports already rather than trying to do something different. Plus Domino developers could benefit from huge communities and technologies with a strong momentum.

      • Smicky

        nsf NodeJS driver : ok ! NodeJS module to authenticate against Domino, ok ! But why embed Node runtime in domino ? (One more component to keep updated). Let’s keep it separated and make Domino “Node friendly” would already be superb.

        But if IBM does not show comitment to the platform (XPages update Java, OSGI, Servlet , NSF etc…all the things the community asks for years) then noone will follow any longer, whatever you add (Node, Cloud or whatever)

        • Paul Withers

          Until recently I don’t think many developers knew about OSGi and Servlet aspects of Domino HTTP server. Even now, I suspect there are very few who have used it, which is going to be the main reason there has not been any development in that area.

          In terms of keeping things separate, IBM tried to do that with using IHS as a web server in front of Domino under 9.0. The POODLE situation showed customers didn’t want a separate HTTP server in front of Domino, which is why so much investment was put into increasing SSL and TLS support on Domino. I’m not sure that expecting a separate node runtime would be a popular approach.

          The best thing IBM can do is allow hooks for the community to take the server the direction they wish, and make it easier for admins to extend. The former has already happened and I have seen initiatives around the latter. In terms of node.js hooks to Domino, there is already activity facilitated by OpenNTF for a project team to make progress on that.

          Personally, though, my preference is to develop on Java, so I’m ambivalent as a developer, but interested as an OpenNTF board member and community member.

  • Shillem Volpato

    Yeah, let’s throw everything at the platform and see what sticks, at this point… what do we have to lose?

    Honestly, this supposed strategy of making Domino be everything can’t be a winning horse. It will be everything and yet nothing: a Windows tablet…

    Supporting js is the least of Domino’s problems. Domino seriously needs focus, that’s its problem.

    On a side note… frankly speaking JavaScript as main language?!? Really? Sure, it’s faster now, broken in 6 months. No, thank you.

  • Nils

    Great points Niklas,

    I have been working on a nodejs driver for nsf in my spare time. It has only very basic functionality at this point. It can read/write/update documents and get views (asynchronous).

    My plan is to make it a opensource project, and hopefully others will be happy to help out. :)

    Making the api similar to Clodant api would be a good idea, I’m going to take a look on that.

  • Niklas Heidloff
  • Adam Brown

    Niklas I believe you are really onto something here. I attended your session at Engage (in fact I am wearing the t-shirt right now:) and have been thinking a lot about it since (that doesn’t happen often with sessions I attend;).

    Anyone that has been around Notes and Domino for a long time knows it is really about the Apps. Yes it has messaging/email etc but the App Dev capabilities has always been why clients have bought and invested in the platform over the competitors. However other than xPages, which has only really had adoption within existing customers and the community, the App Dev strategy has not progressed in recent years. New customers are unlikely to invest in the Notes/Domino platform to get xPages. Existing customers yes, new – unlikely. In addition when going to market for new developers unless they have a history of Notes/Domino rarely does anyone know anything about xPages. Many however know more modern frameworks such as the MEAN Stack.

    Now we know the NSF is very capable database and the replication and access control/security quite unique. If we could simply hook a MEAN app to an NSF, then organisations have a great way to get the best of both worlds. Whether you have node running on Domino is neither here nor there from my perspective. But being able to rapidly integrate MEAN (or your CLEAN) apps with the NSF, have loopback create the CRUD for you, now that would be awesome.

    Your idea of Loopback having a connector for Domino is very exciting if IBM can make it happen, and I believe as you seem to that it would breath new life into the Domino platform.

    • Niklas Heidloff

      Hi Adam, I’m glad you like the t-shirt :)

      Good feedback. It’s a fair question whether or not Domino needs a Node.js runtime or whether easy access from Node apps is sufficient. Michael raised the same concern below.

      I see pros and cons. The reason why I suggest to add a Node runtime to Domino is that you only need to install one environment. I think one of the main reasons why Domino has been so popular is that you get most if not all components you need to build Line of Business apps in one box. That makes app dev on-prem easy since you don’t have to spend time on setting up and configuring different systems.

      On the flip-side it makes things like independent version updates more difficult and it is certainly also more work for IBM to get it well integrated.

      • Adam Brown

        I understand the point. I say convince IBM to do the nsf access from Node apps using Loopback first. That is the wow capability. Then if that succeeds it will be obvious to invest the effort to have node part of Domino as well.

        I really think this is a great option for IBM. I just can’t see xPages becoming mainstream. It is fine for existing customers, but it won’t grow the customer base/community. MEAN is gaining massive traction and if IBM can add unique value around that for nsf then the community can grow. I also think that your thoughts around CLEAN stack provides a potential migration path for clients that are moving away from Domino, rather than just dumping IBM.

  • Philippe Riand

    We can see it from the technical angle or from the business one. Technically, integrating Node.js into a Domino server is pretty easy. Write a DSAPI connector and voila. We proved the feasibility a few years ago using Java, through a prototype called “JSAPI”, for Java integration.

    But the battle is elsewhere, on the business side. Most of the enterprises are reluctant to invest a dime on the platform. I know many BPs who saw their Domino application related business seriously decreasing last year, with some of their big customers stepping out. I’m convinced that adding a new technology to the stack will only benefit a handful of passionate developers, as the participants in this thread. But it won’t help growing the business, or even sustaining it.

    The value of Notes/Domino applications is located in the data and the associated business logic. Nowadays, the solution is more around making these assets available to the new technology stack (Node.js, mobile SDKs, …) rather than getting this technology stack into Domino. The idea is to leave your existing Domino applications running as is, while seamlessly enhancing them using your technology of choice. And eventually fully move to the new stack, over time.

    • Niklas Heidloff

      Phil, Adam and Michael had similar comments around accessing NSF data from other stacks rather than adding more components to the Domino stack. I think it’s a good discussion. A couple of points.

      As you know I’m no longer in ICS and am just sharing some thoughts. The question is whether IBM Collaboration Solutions is ok with “leave your existing Domino applications running as is” or whether the goal is to add modern and non-proprietary app dev capabilities to attract new developers and customers. As mentioned in the comment to Adam I think if you get a full app dev platform when you install ICS products on-prem it adds value since it makes the life of developers and customers easier. For cloud however you get these benefits basically anyway with Bluemix, our platform as a service.

      Personally I’d like developers to be able to use popular frameworks with big communities to build Line of Business and collaborative applications that can integrate with existing Domino applications and ICS products like Connections, Verse and Toscana and that work on-prem as well as in the cloud. That’s why I suggest Node and LoopBack which can actually work against various databases hosted on Bluemix and are very easy to use.

      Regarding “eventually fully move to the new stack, over time” – I hope it’s gonna be another IBM supported stack :) For example Node.js (or Liberty) and Cloudant on Bluemix.

      • http://notesin9.com/ DavidLeedy

        Why would Domino or XPages Developers move to another IBM supported stack? What history do those people have to rely on that IBM is in any way committed to software development? Notes/Domino dev limped along for YEARS then we got the excitement of XPages which was wonderful. But what did they do with that excitement and momentum? Not a lot. 9.0 came out in March of 2013…. 9.0.1 came out in October of 2013. What’s happened since? Not much. A couple fix backs. Bootstrap 3 will have come and been replaced with Bootstrap 4 and it will have never made it into the OFFICIAL product.
        So if people choose to move to another stack over time, I’m at a loss to see why anyone would want to move to an IBM supported stack. Sure Bluemix looks interesting but who knows what IBM will do with it in the future. I think Domino and XPages has suffered because IBM’s focus on Verse, and Bluemix and whatever else. Who’s to say that in a year or two Bluemix will be the step child and IBM will be focusing on something else?

    • Smicky

      I sadly agree with Phil. if IBM does not show extreme comitement to the platform (technically but most of all in marketing, supporting, sales enablement), nothing will happen.

      Phil did a miracle in it’s Time with XPages but support from IBM was too low for the platform to grow.

      Despite that the platform is still extremelly valuable and only suffers from some outdated components that should be refreshed. That + allow other technical stacks to get Domino data and logic and we are ok …provided that IBM shows real love

      Btw Niklas, if customers get rid of Domino because they think IBM neglect them, there are really few chances they stay on an IBM stack…particularily when IBM Bluemix bet on Node, Java, Springboot etc…which all other Cloud providers also provide. This strategy is very strange …

  • http://wfsystems.ca Bill Fox

    I have some suggestions for the basic Domino Designer Client that would make the developer much more efficient. Not sure if this is the place, but here goes.
    Being able to add a category to design objects is really great but needs to be significantly expanded.
    1. Allow a control to have more than one category. In my case my one master XPages database contains the code for multiple applications and I categorize the controls by application,but many controls are common to several different applications.
    2. In the Applications Panel use the categories to group Controls, so you only need to expand the category that you are working on. This would save lots of scrolling through a list of 100 or more controls until you get the one you want.
    3. In the Controls Panel get rid of the accordion and use a ‘categorized view’ like in the Application panel. The accordion works not bad if you don’t have many categories, but is a real pain because with lots of categories they only display a couple entries — more awkward scrolling.

    I realize that this sounds pretty mundane compared to all the really cool new tools, but as developers we work to optimize the end users experience. With a few changes the developers experience could be optimized as well.

  • http://notesin9.com/ DavidLeedy

    I’m all for adding Node.js, IF it doesn’t take away from core product improvements. Honestly I’d probably rather have better connectivity to non .nsf dataSources. then Bluemix would be more viable since you can’t run production .nsf on Bluemix and doesn’t look like you will be able to anytime soon if ever. So I’d much prefer to take the XPages runtime, and use a different Data store.. but that’s just me.

    I get Node.js is the shizzle these days… and yes if it was built into Domino then I’d have a practical means to get started learning it – at least I would know the database etc. That that would be exciting. I just think there are better uses of the resources.

    I do think it’s funny though. It seems that for 20 years IBM has been pushing the Notes Developer to “learn Java”. Pushing hard to use Java back in the day. So now we have XPages. Which is the first really viable way for Domino Developers to realistically and effectively leverage Java.

    So what seems to be the hot topic? Adding Node.js to Domino and telling everybody to go back to JavaScript.

    Sigh….

    • Paul Withers

      “It seems that for 20 years IBM has been pushing the Notes Developer to “learn Java”. Pushing hard to use Java back in the day”. Sorry to disagree, but there’s at least a simple point of evidence that disproves that: when did Domino add an LS UI API (NotesUIWorkspace etc)? Java only got that API in 8.5.1, when those embracing XPages were already looking to move to the web. I’d say the community has pushed developers towards Java in XPages more than IBM.

      • http://notesin9.com/ DavidLeedy

        Paul,
        I don’t have any “evidence” but in my opinion the IBM Java push was real and came WELL before 8.5.1. It was a very constant topic in articles and conferences. Heck there even was a book about Java development in Notes/Domino. There was the constant talk of moving to Websphere, the other Lotus products that non one moved to and whatever else.

    • Niklas Heidloff

      David, don’t forget this is just my personal opinion. Plus I’m not suggesting to replace any of the functionality Domino supports. IBM still focuses on Java, but also on Node.js and now Swift.

  • Karl

    If adding Node.js and OpenWisk to Domino gives the database event triggers we have in the client (document deleted, etc) to missing functionality in Xpage apps then I say go for it. It needs to have good tooling that is integrated into same xpages tooling for rapid development. As far as I know these database type events to NSF data are not triggered in xpages apps. I could be wrong.

    • Karl

      Forgot to mention..OpenWisk looks promising for IoT apps. Enterprises are likely to keep any IoT platform on premise. If enterprises already have Domino on premise it would be a strong selling point for them to stay on the platform..maybe even add customers. IoT and Social have strong ties. I have no idea if this is technically possible, but from a business side it has many good selling points and I like to dream. I would focus on OpenWisk and Node.js secondary just my opinion.

  • Ed Brill

    Hi Niklas, just wanted to thank you for your thought leadership and let you know that as I come back up to speed in this space, I have found this discussion useful. Much appreciated.

  • Steve Bauman

    IBM had the gold in hand and is collapsing .
    Domino was born as NoSQL …20 years ago … it was enough to keep it updated at today .
    Many customers would remain, now customers leave and not return back.

    IBM is not connected with reality and will fail.

    IBM Would push Websphere product which it is mammoth.

    unfortunately we fail expect the future and we will see what will happen.
    I am very sorry to see what’s happening

  • Steve Bauman

    Hello Niklas,

    Your idea to think about Cloudant and XPages runtime is a good idea even if you have to create all the connectors and most likely to redo all our applications based on NSF.
    We need to migrate data from NSF -> Cloudant and recreate a security system based on ACL and rethink agents.
    It is not easy and I say this after 20 years that we work.

    NSF was the first NoSQL in the world and now it comes to these new database.
    If IBM had invested well on this product we would not be where we are now.

    Cloudant is based on CouchDB which was invented by Damien Kiatz, who rewrote the the Notes @Fomulas language.

    I honestly do not understand the benefits of using XPages in bluemix which currently are limited in functionality and honestly I’d like to understand better their future because it is not very clear

    I would like to receive a response from IBM that today seems groped any way out of the fog.

    I hope that my comment is not misplaced and that you do not delete :-)