Why is the topic ID assigned by the server?

Nov 10, 2010 at 8:08 PM

Perhaps there's a usage pattern to this that I'm not seeing, but it would seem to me that it would be most beneficial if the client, in creating a topic, could specify a topic ID to be used, and even more valuable if these topic IDs could be strings.

I'll give you an example. I'd like to use Laharsub to enable many instances of the same application to let each other know when objects have been updated. The way of going about this that makes the most sense to me is for the clients to initialize or subscribe to a Topic with a predictable name. If they are opening the Form for Company 1, then the topic would predictably be "\Company\1" or something of that nature. If they open the form for Sales Order #45271, then a topic for messages reguarding that sales order would be predictably located at \SalesOrder\45271. 

Is there a way to achieve this kind of topic predictability without precreating topics for each entity and storing the related topic ID in the datastore, that perhaps I am just unfamiliar with? 

Nov 11, 2010 at 4:18 AM

I have been asked a similar question in a separate discussion (http://laharsub.codeplex.com/Thread/View.aspx?ThreadId=221086), the relevant fragment is quoted below. I am curious how you would use a feature that enables the client to choose a topic name when creating it: is the intent to facilitate topic discovery across instances of the application, or would you expect some behavioral implications of the topic's hierarchical name structure on the distribution of messages published and\or subscribed to a topic? In the example you provide above it is not clear how the company identifier or the order number is any more discoverable than an opaque topic identifier - in either case I would expect there to be some metadata present outside the context of Laharsub that maps Laharsub's topic names to application specific metadata associated with that topic.

The quote from http://laharsub.codeplex.com/Thread/View.aspx?ThreadId=221086:

I have intentionally designed Laharsub to define a topic only as a unique numeric topic identifier, without attempting to associate any topic metadata or application specific semantics with that identifier. This approach introduces the minimal set of concepts necessary for Laharsub to deliver its core publish/subscribe functionality. At the same time, an application building on top of Laharsub may associate any metadata it needs with a topic identifier. In particular, your stock quote application could associate a ticker symbol with a topic identifier by maintaining a (topicId, ticker) set of tuples in a database or in some static in-memory structure. This application-specific mapping can be consulted by the publisher to decide which topicId to publish messages to when a new pricing data must be distributed for a specific ticker. Similarly, clients can consult this application-specific mapping to decide which topicId should be subscribed to if the user is interested in a particular ticker symbol. In fact, if you clients are web-based, you may provide a web service running alongside Laharsub service that helps clients resolve ticker symbols to topicIds, e.g:

int GetTopicIdForTicker(string ticker);

At some point I was considering an option of associating metadata with a topicId at the time of topic creation. I decided against it since I am not clear what form of the metadata would satisfy the needs of most applications. Some possibilities included a simple list of key-value pairs, a JSON structure, an XML infoset, or an oData query string. Given I don't have a clear picture of the requirements at this point, as well as the availability of an application-level workaround (as described above), I decided to postpone adding such a feature. Opinions will be appreciated.

Nov 11, 2010 at 3:00 PM

To facilitate topic discovery across instances of the application, although there are interesting possibilities with hierarchical message distribution.

Nov 11, 2010 at 4:11 PM
Edited Nov 11, 2010 at 6:06 PM

I've managed to make Laharsub work with string topic ids that are specified by the client. There is no longer a "Create Topic" method- topics are created as soon as a message is published to them. I am now working on a method by which the server can multi-publish a message though a hierarchical topic graph. It is easy to make the client just multi-publish, but this seems like something the server should handle.


http://laharsub.codeplex.com/SourceControl/network/Forks/TDietrich513/Saharsub if you're interested.

Nov 11, 2010 at 7:37 PM

Thank you for experimenting with Laharsub.

What is the scenario where hierarchical message distribution would be useful?

In your approach to creating topics on publish if they don't exist, how do you plan to solve the problem of name conflicts between clients? Or do you think this problem will not exist in case of the applications for which this feature would be useful?

Nov 12, 2010 at 12:53 PM

I'm working on Lab information management software, and one of the features people would like to have is to be notified when a sample they are waiting for analysis on is complete, or when there is more than one person working on the same sample, for the tests entered by other users to show up instantly on their screen. 

So I may create a simple topic graph such as /LABs/Sample.28192/TestSet.19281/Test.82171, where Samples contain many sets of tests which contain many tests. The numbers here would be the the primary keys for each entity.

The User A and User B, working in the same test set would both be subscribed to /LABs/Sample.x/TestSet.x, and User C, who wants to be notified when this sample is complete, would be subscribed to /LABs/Sample.x. 

User A finishes entering results in Test 82171 and completes it. Their client publishes a message to the /LABs/Sample.28192/TestSet.19281/Test.82171 topic. The server then multi-publishes this message to each topic higher up in the graph. So User B with their subscription at the TestSet level gets notified, causing his client to refresh that test, and user C at the Sample level also gets notified.

In this approach, name conflicts are not an issue. The root topic /LABs/ acts as a base namespace to separate different applications using the same Laharsub server, and within each application the primary keys prevent collision.

Make sense?


Nov 12, 2010 at 6:45 PM

I understand your scenario now, thanks for sharing.

I think there are two interesting aspects here: topic aggregation and topic metadata. 

The hierarchical approach to topic aggregation seems sufficient in your particular case, but I wonder if it is generic enough. The hierarchical approach allows for 1-N aggregation of topics. Do you think it is interesting to support N-M aggregation? For example, can a single test belong to more than one test set at the same time? If you had a concept of sample sets, could a single sample belong to more than one sample sets?

Topic metadata is another interesting aspect. With the topic names above you are effectively encoding certain application-specific metadata into the name (the primary keys for samples, test sets, and tests). How does a client application discover these names? How does it know to subscribe to Sample.28192? In a general case the application will need some discovery mechanism to determine the appropriate topic name based on other criteria (e.g. "owner" of a sample, sample title, status, or a completion date), and that discovery mechanism is usually very specific to the application domain. Do you think it would be useful for Laharsub to provide a generic feature that allows topics to tagged with arbitrary metadata (e.g. key value pairs), and searched using those tags?

Nov 12, 2010 at 10:54 PM

In my situation a test only belongs to one test set which only belongs to one sample, so the application can be programmed with certain assumptions. I can see how m..n associations might require a searchable metadata approach rather than this strongly named approach, but the re-broadcast aspect is simple: As topics are created, they could have their parent(s) specified. You'd have to make sure there were no recursive relationships but that's not too bad. 

Jan 31, 2011 at 4:00 PM

Hi, I learned about Laharsub (great app!) when looking for ways for implementing quick notifications for a Silverlight app. I think that using simple strings instead of integers to topic identifiers would be very simple and powerful, regardless of who creates them (client would be nice though).

For prototyping I could use short topics like "im-activity", "temperature", "door134", which I could easily hardcode into clients, without any extra mapping/resolution needed. 

When moving to production (open internet), I could move to namespace-like identifiers to prevent collisions: "Ekus.Dashboard.Temperature", so they would be static and practically globally unique. The exact structure of strings would be up to users/systems, still outside of scope of Laharsub.

The way I have it now on clients is that I have hardcoded integers (topic=30) which are meaningless, and I cannot easily connect if the topic does not exit. So I first have to start 30+ topics on memory-based servers in order to subscribe to the preconfigured "topic". Or add some mapping, which adds complexity.


Jun 19, 2012 at 2:57 PM


Regarding the fork that has been created - Saharsub

Can you please confirm

1) Have you finished the function that does multi-publish?

I have download the code, and i can see the changes made for the creating of a topic when subscribing, but i cannot see the changes regarding the multi-publish