Big Iron and Big Pipes are here!

So let's consolidate some servers!  With the powerful servers and higher bandwidths available today, it makes sense to look at combining several servers into more central locations.  This can reduce costs for maintenance of older boxes, facility costs, and improve overall performance as less time is spent replicating databases between multiple servers.

I recently worked with a client on a consolidation project and want to share some observations.  Their strategy was to build the new servers and, before they moved users to the servers, wanted to test the performance of the central servers.  We used the Server.Load capability that comes with Domino.  This has been around for awhile (I worked with it's predecessor, NotesBench, 15 years ago).  It is an engine that has several scripts that can be used to simulate thousands of users.  It has two basic modes - initialize the environment and running the workload.  The initialization phase is key.  After all, simulating several hundred users is pretty useless.  The challenge of several thousand users is creating that many mail files and populating them with data.

Here is the critical take-away - when running the workload tests, there is a built in delay between each user session which makes sense.  Aside from me, I don't know many users that go flat out all the time.  And this delay is a random number.  The problem is during the initialization phase, you are not collecting statistics, so you don't need to 'simulate' real users, just create the mail files.  But this delay can be up to 15 minutes.  So, doing the math, creating several thousand user mail files with up to 15 minutes between each user can take days (and yes, do use multiple workstations to attack the server).  So, please, help yourself and reduce this delay.  What you need to do is take the built-in initialization script, copy to the clipboard, paste into Notepad, CHANGE THE DELAY, and save the custom script.  Here are the first few lines


* N85 Mail Initialization Workload
* Script to initialize databases for NotesBench N85Mail
* Pause a random interval (0-15 min) so multiple processes are staggered well.
pause 0-900000

Remove at least three zeros from that pause line.  You will be so much happier with the results.

A couple of other comments about server consolidation projects.  A good strategy is to bring up a brand new server.  This means that users will have to re-point their mail files to the new server.  Consider renting a tool like Marvel Client from Panagenda to make this change easier.

Secondly, why 'consolidate' a database that is not being used by a person?  The problem with the activity number in the Catalog is that it considers any activity as using the database.  This activity could be mail being deposited in a database or a scheduled agent running against the database.  Neither of these 'activities' represent a person.  So, while you are ordering the new hardware, consider an approach offered by someone like Inner Ring Solutions.  They have a tool which identify's who is using a database; meaning server, agent, or user; and how often.  So, a database that is not used by users is a candidate to be retired, not consolidated.  Likewise, a database (not mail file) used by 10 or less users could probably be replaced with some other solution, or be advertised to users as an underutilized benefit.  My experience from my days at Teamstudio is that up to 75% of databases in a long standing Notes environment can be deleted because they are not being used.  A large part of this inactivity can be easily identified by looking at you Catalog.  I have a Catalog tool that will identify such things as databases (not templates) with no documents.  Databases with 'test' in the title or file name.  Databases with a year in the title or file name (think 1998 receipts).

If you want more information on your server consolidation project, contact me.  I've been there and done that before, so I can help reduce the pain of your project.

Back in the saddle!

I am so happy to be back working with customers on Lotus Notes development projects!  My 5 years at Teamstudio was a terrific experience (and I would have paid them to do what I did).  But I mostly worked with other developers, and they were always afraid I was trying to sell them something.  But honestly, having worked with Notes for as long as I have, I know the short-comings and why you need tools like Teamstudio, Ytria, Panagenda, etc.  And the prior 7 years I worked for another vendor that nobody heard of which is why I stopped working there.  We sold a very targeted product and were only talking to customers to get them to buy something.

But now, I am back to doing projects that people actually want.  And people are very creative!   And having   had to deal with the state of the economy the last several years, everybody is trying to do more with less.  Everybody has that one little tweak that will solve a whole bunch of headaches.  And those are the kind of challenges I relish.  So, I am back in business.  Contact me if you want another opinion!

It's official!

That small window of opportunity to get my services without any corporate overhead are over.  This week I started working for Greyduck Technology.  Oh, you can still get my services for that project needing extra resources, or even a second opinion/review of your applications and development environment and practices.  It is just that now a portion of my time goes to Greyduck.

I have some great projects already in the works, and am looking forward to working with a company that now has 100 years of Lotus Notes experience on board.

So, stay tuned, and say hello if you see me at your user group.

It takes more programming skill to...

fix someone else's application than to write one from scratch.  At the same time, looking at another developers application will expand your own skill set.  Here is what I mean.  When you stick to your own applications, you tend to stick to what you know.  Your first loop may have been using a counter and GetNthDocument.  Who knew about getfirst - getnext?  And if you looked at some of my later code, the initialize routine only had one line -

Call Main.

(while it might look efficient, it was mostly because only a new subroutine would get the code block that had all my dims in it.)

Looking at other applications will not just show you bad ways of doing things, but other ways of doing things.  When you find an application where everything is defined globally, you begin to appreciate why this is a bad idea.  Trying to understand why the previous developer did things the way they did requires a deeper understanding of coding practices than just writing something from scratch.

So, while Chief Architect may be an alluring title, don't be afraid to take on a less glamorous project involving 'continuing engineering' (my first job title out of college by the way).

And for a limited time, I am available to review your applications without any corporate overhead.  So, drop me a note if you want to take me up on this.

Finding unused views

I recently talked about identifying the views used by users. The bigger challenge is to find those views used programmatically. And, knowing what views are used by users doesn't really help get rid of views because you need a list of all the views in a database. This turns out to be very easy to get - all you need to do is use the NoteCollection class. Here is the relevant code to create a document for every view in a database.

Set nnc = db.CreateNoteCollection( False )
 nnc.Selectviews = True
 
 Call nnc.BuildCollection 
 If nnc.count > 0 Then
  noteID = nnc.GetFirstNoteId 
  
  For x = 1 To nnc.Count
   Set doc = db.Getdocumentbyid( Noteid )
   ' create doc for each element
   Set newDoc = thisDB.Createdocument()
   newDoc.Form = "Views"
   If doc.Hasitem("$Title" ) Then
    set typestring = doc.Getfirstitem("$Title" )
    If InStr( typestring.text, "|" ) Then
     newDoc.Viewname = Left( typestring.text, InStr( typestring.text, "|" )-1 )
     newDoc.Alias = Right( typestring.text, Len( typestring.text) - InStr( typestring.text, "|" ) )
    Else
     newDoc.ViewName = typeString.Text
    End If
   End if
   newDoc.elementNoteID = noteID
   Call newdoc.Save(True, True )
   noteID = nnc.Getnextnoteid( Noteid )
  Next
 End if

This LotusScript was converted to HTML using the ls2html routine,
provided by Julian Robichaux at nsftools.com.

Once you have the list of views, you can now combine this with the list of views used or look in related databases to see who calls the view.  This last part is also very trivial if you use a tool like Teamstudio's Configurator to search all design elements for the view name and aliases.

Let me know if you are interested in this bit as well.

Gamification

Is simply a way to provide positive reinforcement to your users when they use your application the way you want them to.  Do you want them to click the Help button before calling you?  Do you want them to view the Open Actions view more often?  That's where gamification comes in.  The easiest way is to create a script library to handle your counting.  I also use per-user profile documents to track by user.  A simple subroutine would be like this

Sub counter( thing As String )
 Dim s As New NotesSession
 Dim db As NotesDatabase
 Dim doc As NotesDocument
 
 Set db = s.CurrentDatabase
 Set doc = db.GetProfileDocument( "Counter", s.Username )
 If Not( doc.HasItem( thing ) ) Then
  Call doc.Replaceitemvalue( thing, 1 )
 Else
  Call doc.Replaceitemvalue( thing, doc.Getitemvalue( thing )( 0 ) + 1 )
 End If
 Call doc.Save( True, True )
 If ( CDbl( doc.Getitemvalue( thing )) Mod 100 ) = 0 Then
  MessageBox "Thanks for using " + thing + " for " + doc.GetItemValue( thing ) + " times", 48, "Congratulations" 
 End If
End Sub

This LotusScript was converted to HTML using the ls2html routine,
provided by Julian Robichaux at nsftools.com.

Then, in the Click event or PostOpen event for the view (or form or whatever), simply use the script library and add a call to this routine, passing the name of the function (thing) that you want to track.  You can get real creative with this, extending the number of times before a second pop up or whatever.  You can also add in time periods - say for the last month.  It is very low overhead (compared to all the other extra code you might have inherited) and the profile documents are very efficient.

In a related manner, you can use this technique to find out what views users are using.  Simply pass in a view name and use a global profile document for all users.  This is a very powerful way to identify views that users aren't using anymore as candidates for deletion.  This works for user views.  Next post I will describe a technique to use for all those hidden views.

Sadly, this code needs to be added to every view, but I think the results are well worth the time.  I did this for an app with over 200 user views and was well on the way to removing a lot of them.  Maybe someone else has a technique for injecting this code in every view using DXML?

You're paying how much for that server?

Like your cable bill, the internal cost of your servers tends to get bloated over time.  When was the last time you checked your bill?  I was working with one customer whose charges for servers was over $600 K.  Upon looking at the bill, they were paying for servers that were not Domino servers, and paying for servers that didn't exist anymore.

That is a start, but what is the next step?  A good way to consolidate servers is too look at the databases that are being used.  And the key thing is to look at databases not being used by people.  The unfortunate part of the activity reported by the Catalog is it lumps all activity together.  So, you can have a mail file that shows 'activity', but it is simply the router dumping spam in that mail file.  Likewise, you can have an agent running in a database that users don't access.  Thus, activity, but no users.  The good news is that you can use the Activity Logging feature of Domino to generate useful information.  The information provided by Domino is not real useful by itself, but I have a tool that can consolidate the information and give you the information you need to identify databases not being used by people.  Let me know if you want more information.

With this list in hand, you can start to identify servers that can be eliminated by consolidating the databases on fewer servers.  And when you start to consolidate servers, you should think about a tool like Panagenda's Marvel Client that can update all your users desktops to point to the changed server for the databases.