Redfish EventService Implementation

Redfish EventService Implementation

[RAJESWARAN THILLAIGOVINDAN]

Hi,

I am going through the bmcweb code for implementing Redfish EventService 
based on the design document 
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design 
is hooked to the journal based Redfish Event Logging. For life cycle 
events(ResourceAdded, ResourceRemoved, ResourceUpdated),  using D-Bus 
match, bmcweb can create an event log. This requires a JSON dictionary, 
comprising an array of Redfish Resource Name and the D-Bus path. This 
approach works only in case of one to one mapping of Redfish Resource 
Name and the D-Bus path. For propertiesChanged events, if the Redfish 
Resource property is not on the same D-Bus path or the Redfish Resource 
property name is different from the D-Bus property name, then an 
additional JSON dictionary to maintain this information is required. 
With D-Bus match alone in the bmcweb, Redfish EventService can't be 
fully supported. For the Message Registers and the Resource Types that 
are supported, the relevant OpenBMC application must create an event log 
in the journal using either the phosphor::logging::entry or 
sd_journal_send() command.

After realizing that with D-Bus match in the bmcweb alone can't help to 
fully implement EventService, I prefer to avoid using D-Bus match in 
bmcweb. Instead, I prefer to modify the OpenBMC application that 
generated the event to create an event log in the journal. Do you see 
any advantage of using combination of D-Bus match in the bmcweb wherever 
it is possible and changes to OpenBMC application in other cases to 
create an event log ?

Your views are highly appreciated.

Thanks,
Rajes

Re: Redfish EventService Implementation

[RAJESWARAN THILLAIGOVINDAN]

ApparaRao.

As you have shown interest in this feature and submitted the design 
document, do you have any opinion on this? Do you see any merit in using 
D-Bus match in bmcweb to create event logs for life cycle events?  
Please feel free to weigh in.

Thanks,
Rajes

On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
> Hi,
>
> I am going through the bmcweb code for implementing Redfish 
> EventService based on the design document 
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design 
> is hooked to the journal based Redfish Event Logging. For life cycle 
> events(ResourceAdded, ResourceRemoved, ResourceUpdated),  using D-Bus 
> match, bmcweb can create an event log. This requires a JSON 
> dictionary, comprising an array of Redfish Resource Name and the D-Bus 
> path. This approach works only in case of one to one mapping of 
> Redfish Resource Name and the D-Bus path. For propertiesChanged 
> events, if the Redfish Resource property is not on the same D-Bus path 
> or the Redfish Resource property name is different from the D-Bus 
> property name, then an additional JSON dictionary to maintain this 
> information is required. With D-Bus match alone in the bmcweb, Redfish 
> EventService can't be fully supported. For the Message Registers and 
> the Resource Types that are supported, the relevant OpenBMC 
> application must create an event log in the journal using either the 
> phosphor::logging::entry or sd_journal_send() command.
>
> After realizing that with D-Bus match in the bmcweb alone can't help 
> to fully implement EventService, I prefer to avoid using D-Bus match 
> in bmcweb. Instead, I prefer to modify the OpenBMC application that 
> generated the event to create an event log in the journal. Do you see 
> any advantage of using combination of D-Bus match in the bmcweb 
> wherever it is possible and changes to OpenBMC application in other 
> cases to create an event log ?
>
> Your views are highly appreciated.
>
> Thanks,
> Rajes

Re: Redfish EventService Implementation

[RAJESWARAN THILLAIGOVINDAN]

I have implemented the skeleton code and uploaded it for review. Kindly 
review and let me know your comments.  For prototyping, based on the 
Redfish Event Logging design, I have modified phosphosr-user-manager 
application to log resource creation event when an account is created. 
For reading the redfish event logs from the journal and writing to BMC 
filesystem(/var/log/redfish), I have pulled the rsyslog configuration 
from the OEM (Intel) and made minor changes.

These patches are work in progress and needs lot more changes. Any 
suggestions regarding the approach is appreciated.

https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/29464
https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-user-manager/+/29463
https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/29462

Thanks,
Rajes
On 10-02-2020 01:52, RAJESWARAN THILLAIGOVINDAN wrote:
> ApparaRao.
>
> As you have shown interest in this feature and submitted the design 
> document, do you have any opinion on this? Do you see any merit in 
> using D-Bus match in bmcweb to create event logs for life cycle 
> events?  Please feel free to weigh in.
>
> Thanks,
> Rajes
>
> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>> Hi,
>>
>> I am going through the bmcweb code for implementing Redfish 
>> EventService based on the design document 
>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This 
>> design is hooked to the journal based Redfish Event Logging. For life 
>> cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated),  using 
>> D-Bus match, bmcweb can create an event log. This requires a JSON 
>> dictionary, comprising an array of Redfish Resource Name and the 
>> D-Bus path. This approach works only in case of one to one mapping of 
>> Redfish Resource Name and the D-Bus path. For propertiesChanged 
>> events, if the Redfish Resource property is not on the same D-Bus 
>> path or the Redfish Resource property name is different from the 
>> D-Bus property name, then an additional JSON dictionary to maintain 
>> this information is required. With D-Bus match alone in the bmcweb, 
>> Redfish EventService can't be fully supported. For the Message 
>> Registers and the Resource Types that are supported, the relevant 
>> OpenBMC application must create an event log in the journal using 
>> either the phosphor::logging::entry or sd_journal_send() command.
>>
>> After realizing that with D-Bus match in the bmcweb alone can't help 
>> to fully implement EventService, I prefer to avoid using D-Bus match 
>> in bmcweb. Instead, I prefer to modify the OpenBMC application that 
>> generated the event to create an event log in the journal. Do you see 
>> any advantage of using combination of D-Bus match in the bmcweb 
>> wherever it is possible and changes to OpenBMC application in other 
>> cases to create an event log ?
>>
>> Your views are highly appreciated.
>>
>> Thanks,
>> Rajes
>

Re: Redfish EventService Implementation

[Puli, Apparao]

Hi,

   I am sorry for late response as this mail is buried under and got 
struck back of my mind.

As i did mentioned in EventService design document, EventLog Monitor 
service is not common across the organizations( Example: Intel uses the 
redfish event logs file and inotify mechanism for monitoring the event 
logs. Where as IBM uses d-bus event log mechanism and they can relay on 
match rules). That said challenges with ResourceType mapping will be 
different in both monitoring mechanisms. This is good point. Initially 
when i started EventService design, i thought we can have mapping in 
bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
per Intel design) but not sure that may become difficult when we expand 
supported ResourceTypes.

As per my reading from below query, You are looking at d-bus match rules 
and ResourceTypes mapping which is more specific to d-bus event 
logging(IBM way of implementing event logging). reading it from journal 
logs will give more information but that will impact the performance to 
large extent. This might be one of the reason why we (Intel) uses 
Redfish message ID while logging redfish events logs to journal(You can 
refer design document for same at 
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
In opinion, in your d-bus if you are using some kind of filter(Example 
REDFISH_MESSAGE_ID) while logging in journal logs for all events and 
figure out the way to monitor the journal logs without impacting the 
performance, that should be ok as long as match filters are satisfied 
for Redfish EventService subscriptions and supported Types(Again differs 
with implementation).

Thanks,

-Appu

On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
> ApparaRao.
>
> As you have shown interest in this feature and submitted the design 
> document, do you have any opinion on this? Do you see any merit in 
> using D-Bus match in bmcweb to create event logs for life cycle 
> events?  Please feel free to weigh in.
>
> Thanks,
> Rajes
>
> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>> Hi,
>>
>> I am going through the bmcweb code for implementing Redfish 
>> EventService based on the design document 
>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This 
>> design is hooked to the journal based Redfish Event Logging. For life 
>> cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated),  using 
>> D-Bus match, bmcweb can create an event log. This requires a JSON 
>> dictionary, comprising an array of Redfish Resource Name and the 
>> D-Bus path. This approach works only in case of one to one mapping of 
>> Redfish Resource Name and the D-Bus path. For propertiesChanged 
>> events, if the Redfish Resource property is not on the same D-Bus 
>> path or the Redfish Resource property name is different from the 
>> D-Bus property name, then an additional JSON dictionary to maintain 
>> this information is required. With D-Bus match alone in the bmcweb, 
>> Redfish EventService can't be fully supported. For the Message 
>> Registers and the Resource Types that are supported, the relevant 
>> OpenBMC application must create an event log in the journal using 
>> either the phosphor::logging::entry or sd_journal_send() command.
>>
>> After realizing that with D-Bus match in the bmcweb alone can't help 
>> to fully implement EventService, I prefer to avoid using D-Bus match 
>> in bmcweb. Instead, I prefer to modify the OpenBMC application that 
>> generated the event to create an event log in the journal. Do you see 
>> any advantage of using combination of D-Bus match in the bmcweb 
>> wherever it is possible and changes to OpenBMC application in other 
>> cases to create an event log ?
>>
>> Your views are highly appreciated.
>>
>> Thanks,
>> Rajes
>

Re: Redfish EventService Implementation

[Ratan Gupta]

[-- Attachment #1: Type: text/plain, Size: 5454 bytes --]

Hi Apparao,

On 2/20/20 12:49 AM, Puli, Apparao wrote:
> Hi,
>
>   I am sorry for late response as this mail is buried under and got 
> struck back of my mind.
>
> As i did mentioned in EventService design document, EventLog Monitor 
> service is not common across the organizations( Example: Intel uses 
> the redfish event logs file and inotify mechanism for monitoring the 
> event logs. Where as IBM uses d-bus event log mechanism and they can 
> relay on match rules). That said challenges with ResourceType mapping 
> will be different in both monitoring mechanisms. This is good point. 
> Initially when i started EventService design, i thought we can have 
> mapping in bmcweb for ResourceTypes with set of MessageID's for Logged 
> events ( As per Intel design) but not sure that may become difficult 
> when we expand supported ResourceTypes.

If I am getting correctly, Here is the flow which Intel uses.

 1. Individual repo have to push the logs using sd_journal_send which
    will write to the file(/var/log/redfish) by using rsyslog daemon

sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
                 "REDFISH_MESSAGE_ID=%s",
                 "ResourceEvent.1.0.ResourceCreated",NULL);

      * How you would populate the "OriginOfCondition" during sending of
        event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)

      * Any plan to add resource path in this journal message which
        tells that this is the path for which resource created event
        generated.

      * Would the path be "Redfish Path/ D-bus Path"?

      * Where the mapping would be done between D-busPath/Redfish
        Resource path?

     
          Cons: Every application have to make the change(use sd_journal_send).
          My thought is backend application should not be aware of the redfish terminlogy.

*2.* Some application(bmcweb) would do the Inotify on the 
path(/var/log/redfish) and send the event once there is any activity on 
this file.

> I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
per Intel design)

     Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.

>
> As per my reading from below query, You are looking at d-bus match 
> rules and ResourceTypes mapping which is more specific to d-bus event 
> logging(IBM way of implementing event logging). reading it from 
> journal logs will give more information but that will impact the 
> performance to large extent. This might be one of the reason why we 
> (Intel) uses Redfish message ID while logging redfish events logs to 
> journal(You can refer design document for same at 
> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
> In opinion, in your d-bus if you are using some kind of filter(Example 
> REDFISH_MESSAGE_ID) while logging in journal logs for all events and 
> figure out the way to monitor the journal logs without impacting the 
> performance, that should be ok as long as match filters are satisfied 
> for Redfish EventService subscriptions and supported Types(Again 
> differs with implementation).
>
> Thanks,
>
> -Appu
>
> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>> ApparaRao.
>>
>> As you have shown interest in this feature and submitted the design 
>> document, do you have any opinion on this? Do you see any merit in 
>> using D-Bus match in bmcweb to create event logs for life cycle 
>> events?  Please feel free to weigh in.
>>
>> Thanks,
>> Rajes
>>
>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>> Hi,
>>>
>>> I am going through the bmcweb code for implementing Redfish 
>>> EventService based on the design document 
>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This 
>>> design is hooked to the journal based Redfish Event Logging. For 
>>> life cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated),  
>>> using D-Bus match, bmcweb can create an event log. This requires a 
>>> JSON dictionary, comprising an array of Redfish Resource Name and 
>>> the D-Bus path. This approach works only in case of one to one 
>>> mapping of Redfish Resource Name and the D-Bus path. For 
>>> propertiesChanged events, if the Redfish Resource property is not on 
>>> the same D-Bus path or the Redfish Resource property name is 
>>> different from the D-Bus property name, then an additional JSON 
>>> dictionary to maintain this information is required. With D-Bus 
>>> match alone in the bmcweb, Redfish EventService can't be fully 
>>> supported. For the Message Registers and the Resource Types that are 
>>> supported, the relevant OpenBMC application must create an event log 
>>> in the journal using either the phosphor::logging::entry or 
>>> sd_journal_send() command.
>>>
>>> After realizing that with D-Bus match in the bmcweb alone can't help 
>>> to fully implement EventService, I prefer to avoid using D-Bus match 
>>> in bmcweb. Instead, I prefer to modify the OpenBMC application that 
>>> generated the event to create an event log in the journal. Do you 
>>> see any advantage of using combination of D-Bus match in the bmcweb 
>>> wherever it is possible and changes to OpenBMC application in other 
>>> cases to create an event log ?
>>>
>>> Your views are highly appreciated.
>>>
>>> Thanks,
>>> Rajes
>>
Thanks
Ratan



[-- Attachment #2: Type: text/html, Size: 8088 bytes --]

Re: Redfish EventService Implementation

[Puli, Apparao]

[-- Attachment #1: Type: text/plain, Size: 7102 bytes --]

Hi Ratan,

    Comments in-line

On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>
> Hi Apparao,
>
> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>> Hi,
>>
>>   I am sorry for late response as this mail is buried under and got 
>> struck back of my mind.
>>
>> As i did mentioned in EventService design document, EventLog Monitor 
>> service is not common across the organizations( Example: Intel uses 
>> the redfish event logs file and inotify mechanism for monitoring the 
>> event logs. Where as IBM uses d-bus event log mechanism and they can 
>> relay on match rules). That said challenges with ResourceType mapping 
>> will be different in both monitoring mechanisms. This is good point. 
>> Initially when i started EventService design, i thought we can have 
>> mapping in bmcweb for ResourceTypes with set of MessageID's for 
>> Logged events ( As per Intel design) but not sure that may become 
>> difficult when we expand supported ResourceTypes.
>
> If I am getting correctly, Here is the flow which Intel uses.
>
>  1. Individual repo have to push the logs using sd_journal_send which
>     will write to the file(/var/log/redfish) by using rsyslog daemon
>
> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>                  "REDFISH_MESSAGE_ID=%s",
>                  "ResourceEvent.1.0.ResourceCreated",NULL);
>
>       * How you would populate the "OriginOfCondition" during sending
>         of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>
Currently in logServices( logEntry),  we are not reporting the 
"OriginOfCondition" property as per schema. I will check with Jason( Who 
wrote the logService) and get back on this.

BTW can you give me how this information is fetched in IBM way of 
LogService implementation( D-Bus)? If you already ratified any such, i 
think we can leverage.  If not, We work together for solution.

>       * Any plan to add resource path in this journal message which
>         tells that this is the path for which resource created event
>         generated.
>
Same as above.
>
>       * Would the path be "Redfish Path/ D-bus Path"?
>
As per Redfish specification, This should be "@odata.id" which means it 
should be of resource uri and so we can't use d-bus path here for 
OriginOfConditions.
>
>       * Where the mapping would be done between D-busPath/Redfish
>         Resource path?
>
>      
>           Cons: Every application have to make the change(use sd_journal_send).
>           My thought is backend application should not be aware of the redfish terminlogy.

Having separate process only for this mapping may not be good( No 
different from maintaining that map inside bmcweb as there won't be any 
other consumers). Ideal way is, that should be mapped while logging 
logEntry's itself. But we are not doing it currently which need to be 
re-looked. Give me some time, I will think and check with other folks 
and get back.

>
> *2.* Some application(bmcweb) would do the Inotify on the 
> path(/var/log/redfish) and send the event once there is any activity 
> on this file.
>
> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
> per Intel design)
>
>      Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done 
using minimal mapping( Using MessageID and args). But that will work for 
minimal set. If the supported ResourceTypes grows, we will have bigger 
challenges which i can sense it now.  Anyway, Supported Resources are 
completely implementation specific. If this value is empty means, by 
default all event logs will be sent to subscribers. This is what we can 
start with before supported  ResourceTypes list grows.
>>
>> As per my reading from below query, You are looking at d-bus match 
>> rules and ResourceTypes mapping which is more specific to d-bus event 
>> logging(IBM way of implementing event logging). reading it from 
>> journal logs will give more information but that will impact the 
>> performance to large extent. This might be one of the reason why we 
>> (Intel) uses Redfish message ID while logging redfish events logs to 
>> journal(You can refer design document for same at 
>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
>> In opinion, in your d-bus if you are using some kind of 
>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for 
>> all events and figure out the way to monitor the journal logs without 
>> impacting the performance, that should be ok as long as match filters 
>> are satisfied for Redfish EventService subscriptions and supported 
>> Types(Again differs with implementation).
>>
>> Thanks,
>>
>> -Appu
>>
>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>> ApparaRao.
>>>
>>> As you have shown interest in this feature and submitted the design 
>>> document, do you have any opinion on this? Do you see any merit in 
>>> using D-Bus match in bmcweb to create event logs for life cycle 
>>> events?  Please feel free to weigh in.
>>>
>>> Thanks,
>>> Rajes
>>>
>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>> Hi,
>>>>
>>>> I am going through the bmcweb code for implementing Redfish 
>>>> EventService based on the design document 
>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This 
>>>> design is hooked to the journal based Redfish Event Logging. For 
>>>> life cycle events(ResourceAdded, ResourceRemoved, 
>>>> ResourceUpdated),  using D-Bus match, bmcweb can create an event 
>>>> log. This requires a JSON dictionary, comprising an array of 
>>>> Redfish Resource Name and the D-Bus path. This approach works only 
>>>> in case of one to one mapping of Redfish Resource Name and the 
>>>> D-Bus path. For propertiesChanged events, if the Redfish Resource 
>>>> property is not on the same D-Bus path or the Redfish Resource 
>>>> property name is different from the D-Bus property name, then an 
>>>> additional JSON dictionary to maintain this information is 
>>>> required. With D-Bus match alone in the bmcweb, Redfish 
>>>> EventService can't be fully supported. For the Message Registers 
>>>> and the Resource Types that are supported, the relevant OpenBMC 
>>>> application must create an event log in the journal using either 
>>>> the phosphor::logging::entry or sd_journal_send() command.
>>>>
>>>> After realizing that with D-Bus match in the bmcweb alone can't 
>>>> help to fully implement EventService, I prefer to avoid using D-Bus 
>>>> match in bmcweb. Instead, I prefer to modify the OpenBMC 
>>>> application that generated the event to create an event log in the 
>>>> journal. Do you see any advantage of using combination of D-Bus 
>>>> match in the bmcweb wherever it is possible and changes to OpenBMC 
>>>> application in other cases to create an event log ?
>>>>
>>>> Your views are highly appreciated.
>>>>
>>>> Thanks,
>>>> Rajes
>>>
> Thanks
> Ratan
>
>

[-- Attachment #2: Type: text/html, Size: 11239 bytes --]

Re: Redfish EventService Implementation

[RAJESWARAN THILLAIGOVINDAN]

[-- Attachment #1: Type: text/plain, Size: 8540 bytes --]


Further to my previous mail, I wanted to see how a JSON dictionary would 
look like for implementing Redfish EventService based on D-Bus match. 
The D-Bus match approach is good for implementing life cycle 
events((ResourceAdded, ResourceRemoved, ResourceUpdated). In bmcweb, a 
Redfish Resource is viewed as having a set of properties which comes 
from one or more D-Bus objects. So, when a client subscribes for a 
ResourceType, using a JSON dictionary, find the D-Bus object(s) mapped 
to the ResourceType and create matches. When the D-Bus match occurs, 
again using aJSON dictionary, find the Redfish ResourceType mapped to 
the D-Bus object and create events. The JSON dictionary should also 
provide the Redifsh URI which needs to be specified in the event. An 
example JSON dictionary for mapping the Redifsh ResourceTypes LogEntry 
and ComputerSystem is available here : 
https://gist.github.com/trajeswaran/fec230abd36181f85d2f20d09164ec05. In 
case of LogEntry, there is one to one mapping of ResourceType to D-Bus 
object. In case of ComputerSystem, the properties comes from multiple 
D-Bus objects. Do you see any drawback with this approach?  Kindly let 
me know what you think. Thanks in advance.


On 25-02-2020 19:36, Puli, Apparao wrote:
>
> Hi Ratan,
>
>    Comments in-line
>
> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>
>> Hi Apparao,
>>
>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>> Hi,
>>>
>>>   I am sorry for late response as this mail is buried under and got 
>>> struck back of my mind.
>>>
>>> As i did mentioned in EventService design document, EventLog Monitor 
>>> service is not common across the organizations( Example: Intel uses 
>>> the redfish event logs file and inotify mechanism for monitoring the 
>>> event logs. Where as IBM uses d-bus event log mechanism and they can 
>>> relay on match rules). That said challenges with ResourceType 
>>> mapping will be different in both monitoring mechanisms. This is 
>>> good point. Initially when i started EventService design, i thought 
>>> we can have mapping in bmcweb for ResourceTypes with set of 
>>> MessageID's for Logged events ( As per Intel design) but not sure 
>>> that may become difficult when we expand supported ResourceTypes.
>>
>> If I am getting correctly, Here is the flow which Intel uses.
>>
>>  1. Individual repo have to push the logs using sd_journal_send which
>>     will write to the file(/var/log/redfish) by using rsyslog daemon
>>
>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>                  "REDFISH_MESSAGE_ID=%s",
>>                  "ResourceEvent.1.0.ResourceCreated",NULL);
>>
>>       * How you would populate the "OriginOfCondition" during sending
>>         of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>
> Currently in logServices( logEntry),  we are not reporting the 
> "OriginOfCondition" property as per schema. I will check with Jason( 
> Who wrote the logService) and get back on this.
>
> BTW can you give me how this information is fetched in IBM way of 
> LogService implementation( D-Bus)? If you already ratified any such, i 
> think we can leverage.  If not, We work together for solution.
>
>>       * Any plan to add resource path in this journal message which
>>         tells that this is the path for which resource created event
>>         generated.
>>
> Same as above.
>>
>>       * Would the path be "Redfish Path/ D-bus Path"?
>>
> As per Redfish specification, This should be "@odata.id" which means 
> it should be of resource uri and so we can't use d-bus path here for 
> OriginOfConditions.
>>
>>       * Where the mapping would be done between D-busPath/Redfish
>>         Resource path?
>>
>>      
>>           Cons: Every application have to make the change(use sd_journal_send).
>>           My thought is backend application should not be aware of the redfish terminlogy.
>
> Having separate process only for this mapping may not be good( No 
> different from maintaining that map inside bmcweb as there won't be 
> any other consumers). Ideal way is, that should be mapped while 
> logging logEntry's itself. But we are not doing it currently which 
> need to be re-looked. Give me some time, I will think and check with 
> other folks and get back.
>
>> *2.* Some application(bmcweb) would do the Inotify on the 
>> path(/var/log/redfish) and send the event once there is any activity 
>> on this file.
>>
>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
>> per Intel design)
>>
>>      Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
> Initially i thought "ResourceType" based event filtering can be done 
> using minimal mapping( Using MessageID and args). But that will work 
> for minimal set. If the supported ResourceTypes grows, we will have 
> bigger challenges which i can sense it now.  Anyway, Supported 
> Resources are completely implementation specific. If this value is 
> empty means, by default all event logs will be sent to subscribers. 
> This is what we can start with before supported ResourceTypes list grows.
>>>
>>> As per my reading from below query, You are looking at d-bus match 
>>> rules and ResourceTypes mapping which is more specific to d-bus 
>>> event logging(IBM way of implementing event logging). reading it 
>>> from journal logs will give more information but that will impact 
>>> the performance to large extent. This might be one of the reason why 
>>> we (Intel) uses Redfish message ID while logging redfish events logs 
>>> to journal(You can refer design document for same at 
>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
>>> In opinion, in your d-bus if you are using some kind of 
>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for 
>>> all events and figure out the way to monitor the journal logs 
>>> without impacting the performance, that should be ok as long as 
>>> match filters are satisfied for Redfish EventService subscriptions 
>>> and supported Types(Again differs with implementation).
>>>
>>> Thanks,
>>>
>>> -Appu
>>>
>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>> ApparaRao.
>>>>
>>>> As you have shown interest in this feature and submitted the design 
>>>> document, do you have any opinion on this? Do you see any merit in 
>>>> using D-Bus match in bmcweb to create event logs for life cycle 
>>>> events?  Please feel free to weigh in.
>>>>
>>>> Thanks,
>>>> Rajes
>>>>
>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>> Hi,
>>>>>
>>>>> I am going through the bmcweb code for implementing Redfish 
>>>>> EventService based on the design document 
>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This 
>>>>> design is hooked to the journal based Redfish Event Logging. For 
>>>>> life cycle events(ResourceAdded, ResourceRemoved, 
>>>>> ResourceUpdated),  using D-Bus match, bmcweb can create an event 
>>>>> log. This requires a JSON dictionary, comprising an array of 
>>>>> Redfish Resource Name and the D-Bus path. This approach works only 
>>>>> in case of one to one mapping of Redfish Resource Name and the 
>>>>> D-Bus path. For propertiesChanged events, if the Redfish Resource 
>>>>> property is not on the same D-Bus path or the Redfish Resource 
>>>>> property name is different from the D-Bus property name, then an 
>>>>> additional JSON dictionary to maintain this information is 
>>>>> required. With D-Bus match alone in the bmcweb, Redfish 
>>>>> EventService can't be fully supported. For the Message Registers 
>>>>> and the Resource Types that are supported, the relevant OpenBMC 
>>>>> application must create an event log in the journal using either 
>>>>> the phosphor::logging::entry or sd_journal_send() command.
>>>>>
>>>>> After realizing that with D-Bus match in the bmcweb alone can't 
>>>>> help to fully implement EventService, I prefer to avoid using 
>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC 
>>>>> application that generated the event to create an event log in the 
>>>>> journal. Do you see any advantage of using combination of D-Bus 
>>>>> match in the bmcweb wherever it is possible and changes to OpenBMC 
>>>>> application in other cases to create an event log ?
>>>>>
>>>>> Your views are highly appreciated.
>>>>>
>>>>> Thanks,
>>>>> Rajes
>>>>
>> Thanks
>> Ratan
>>
>>

[-- Attachment #2: Type: text/html, Size: 13434 bytes --]

Re: Redfish EventService Implementation

[RAJESWARAN THILLAIGOVINDAN]

[-- Attachment #1: Type: text/plain, Size: 8212 bytes --]

Apparao,

I see that you have uploaded a 
patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for 
supporting server sent events. This patch supports event filtering based 
on registry prefix  and/or messageId.

I would like to know if you have any plan to support event filtering 
based on resource type. If so, I would like to work together for a 
better solution. Earlier, I have proposed a solution based on D-Bus 
match using a dictionary. With that approach, the major challenge is to 
map Redfish resource and properties to D-Bus object and properties 
respectively.   If D-Bus applications are modified to include resource 
type and origin of condition in the event, then there is no need for a 
map. But,that brings Redfish terminology to the application. Also, this 
will become an overhead if an OEM is not interested in Redfish event 
service.

Thanks,
Rajes
On 25-02-2020 19:36, Puli, Apparao wrote:
>
> Hi Ratan,
>
>    Comments in-line
>
> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>
>> Hi Apparao,
>>
>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>> Hi,
>>>
>>>   I am sorry for late response as this mail is buried under and got 
>>> struck back of my mind.
>>>
>>> As i did mentioned in EventService design document, EventLog Monitor 
>>> service is not common across the organizations( Example: Intel uses 
>>> the redfish event logs file and inotify mechanism for monitoring the 
>>> event logs. Where as IBM uses d-bus event log mechanism and they can 
>>> relay on match rules). That said challenges with ResourceType 
>>> mapping will be different in both monitoring mechanisms. This is 
>>> good point. Initially when i started EventService design, i thought 
>>> we can have mapping in bmcweb for ResourceTypes with set of 
>>> MessageID's for Logged events ( As per Intel design) but not sure 
>>> that may become difficult when we expand supported ResourceTypes.
>>
>> If I am getting correctly, Here is the flow which Intel uses.
>>
>>  1. Individual repo have to push the logs using sd_journal_send which
>>     will write to the file(/var/log/redfish) by using rsyslog daemon
>>
>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>                  "REDFISH_MESSAGE_ID=%s",
>>                  "ResourceEvent.1.0.ResourceCreated",NULL);
>>
>>       * How you would populate the "OriginOfCondition" during sending
>>         of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>
> Currently in logServices( logEntry),  we are not reporting the 
> "OriginOfCondition" property as per schema. I will check with Jason( 
> Who wrote the logService) and get back on this.
>
> BTW can you give me how this information is fetched in IBM way of 
> LogService implementation( D-Bus)? If you already ratified any such, i 
> think we can leverage.  If not, We work together for solution.
>
>>       * Any plan to add resource path in this journal message which
>>         tells that this is the path for which resource created event
>>         generated.
>>
> Same as above.
>>
>>       * Would the path be "Redfish Path/ D-bus Path"?
>>
> As per Redfish specification, This should be "@odata.id" which means 
> it should be of resource uri and so we can't use d-bus path here for 
> OriginOfConditions.
>>
>>       * Where the mapping would be done between D-busPath/Redfish
>>         Resource path?
>>
>>      
>>           Cons: Every application have to make the change(use sd_journal_send).
>>           My thought is backend application should not be aware of the redfish terminlogy.
>
> Having separate process only for this mapping may not be good( No 
> different from maintaining that map inside bmcweb as there won't be 
> any other consumers). Ideal way is, that should be mapped while 
> logging logEntry's itself. But we are not doing it currently which 
> need to be re-looked. Give me some time, I will think and check with 
> other folks and get back.
>
>> *2.* Some application(bmcweb) would do the Inotify on the 
>> path(/var/log/redfish) and send the event once there is any activity 
>> on this file.
>>
>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
>> per Intel design)
>>
>>      Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
> Initially i thought "ResourceType" based event filtering can be done 
> using minimal mapping( Using MessageID and args). But that will work 
> for minimal set. If the supported ResourceTypes grows, we will have 
> bigger challenges which i can sense it now.  Anyway, Supported 
> Resources are completely implementation specific. If this value is 
> empty means, by default all event logs will be sent to subscribers. 
> This is what we can start with before supported ResourceTypes list grows.
>>>
>>> As per my reading from below query, You are looking at d-bus match 
>>> rules and ResourceTypes mapping which is more specific to d-bus 
>>> event logging(IBM way of implementing event logging). reading it 
>>> from journal logs will give more information but that will impact 
>>> the performance to large extent. This might be one of the reason why 
>>> we (Intel) uses Redfish message ID while logging redfish events logs 
>>> to journal(You can refer design document for same at 
>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
>>> In opinion, in your d-bus if you are using some kind of 
>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for 
>>> all events and figure out the way to monitor the journal logs 
>>> without impacting the performance, that should be ok as long as 
>>> match filters are satisfied for Redfish EventService subscriptions 
>>> and supported Types(Again differs with implementation).
>>>
>>> Thanks,
>>>
>>> -Appu
>>>
>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>> ApparaRao.
>>>>
>>>> As you have shown interest in this feature and submitted the design 
>>>> document, do you have any opinion on this? Do you see any merit in 
>>>> using D-Bus match in bmcweb to create event logs for life cycle 
>>>> events?  Please feel free to weigh in.
>>>>
>>>> Thanks,
>>>> Rajes
>>>>
>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>> Hi,
>>>>>
>>>>> I am going through the bmcweb code for implementing Redfish 
>>>>> EventService based on the design document 
>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This 
>>>>> design is hooked to the journal based Redfish Event Logging. For 
>>>>> life cycle events(ResourceAdded, ResourceRemoved, 
>>>>> ResourceUpdated),  using D-Bus match, bmcweb can create an event 
>>>>> log. This requires a JSON dictionary, comprising an array of 
>>>>> Redfish Resource Name and the D-Bus path. This approach works only 
>>>>> in case of one to one mapping of Redfish Resource Name and the 
>>>>> D-Bus path. For propertiesChanged events, if the Redfish Resource 
>>>>> property is not on the same D-Bus path or the Redfish Resource 
>>>>> property name is different from the D-Bus property name, then an 
>>>>> additional JSON dictionary to maintain this information is 
>>>>> required. With D-Bus match alone in the bmcweb, Redfish 
>>>>> EventService can't be fully supported. For the Message Registers 
>>>>> and the Resource Types that are supported, the relevant OpenBMC 
>>>>> application must create an event log in the journal using either 
>>>>> the phosphor::logging::entry or sd_journal_send() command.
>>>>>
>>>>> After realizing that with D-Bus match in the bmcweb alone can't 
>>>>> help to fully implement EventService, I prefer to avoid using 
>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC 
>>>>> application that generated the event to create an event log in the 
>>>>> journal. Do you see any advantage of using combination of D-Bus 
>>>>> match in the bmcweb wherever it is possible and changes to OpenBMC 
>>>>> application in other cases to create an event log ?
>>>>>
>>>>> Your views are highly appreciated.
>>>>>
>>>>> Thanks,
>>>>> Rajes
>>>>
>> Thanks
>> Ratan
>>
>>

[-- Attachment #2: Type: text/html, Size: 13084 bytes --]

Re: Redfish EventService Implementation

[Puli, Apparao]

[-- Attachment #1: Type: text/plain, Size: 9633 bytes --]

Hi Rajeswaran,

   Thanks for your mail. At the moment, I don't have plans to support 
the "ResourceTypes", "OriginResources" based filtering. Basically Intel 
uses file systems based redfish event logs( journalctl -> rsync-> 
filesystem) and doesn't use D-Bus mechanism like IBM uses. So I am not 
much familiar with D-Bus logging but some of the suggestions:

  1) While logging redfish events over D-Bus itself,  it can provide 
details on ResourceTypes and OriginResource URI/Path.

      This is ideal and most efficient way. Since  we walked a walked 
long distance from start, Its hard to modify all the services to uses 
these 2 new input parameters while logging events( Requires change in 
almost all repo's)

2) For resourcesTypes: Can have mapping dictionary against all 
MessageId's. For OriginResources: I believe, event log over D-Bus is 
already holding the Path. If so, last 3/4 nodes of uri can be taken and 
mapped against the resources and that can be used in Event filtering. We 
did used same mechanism in case of telemetry while mapping 
MetricReportDefinitions to URI.

Hope this helps.

Thanks,

-Appu


On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>
> Apparao,
>
> I see that you have uploaded a 
> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for 
> supporting server sent events. This patch supports event filtering 
> based on registry prefix  and/or messageId.
>
> I would like to know if you have any plan to support event filtering 
> based on resource type. If so, I would like to work together for a 
> better solution. Earlier, I have proposed a solution based on D-Bus 
> match using a dictionary. With that approach, the major challenge is 
> to map Redfish resource and properties to D-Bus object and properties 
> respectively.   If D-Bus applications are modified to include resource 
> type and origin of condition in the event, then there is no need for a 
> map. But,that brings Redfish terminology to the application. Also, 
> this will become an overhead if an OEM is not interested in Redfish 
> event service.
>
> Thanks,
> Rajes
> On 25-02-2020 19:36, Puli, Apparao wrote:
>>
>> Hi Ratan,
>>
>>    Comments in-line
>>
>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>
>>> Hi Apparao,
>>>
>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>> Hi,
>>>>
>>>>   I am sorry for late response as this mail is buried under and got 
>>>> struck back of my mind.
>>>>
>>>> As i did mentioned in EventService design document, EventLog 
>>>> Monitor service is not common across the organizations( Example: 
>>>> Intel uses the redfish event logs file and inotify mechanism for 
>>>> monitoring the event logs. Where as IBM uses d-bus event log 
>>>> mechanism and they can relay on match rules). That said challenges 
>>>> with ResourceType mapping will be different in both monitoring 
>>>> mechanisms. This is good point. Initially when i started 
>>>> EventService design, i thought we can have mapping in bmcweb for 
>>>> ResourceTypes with set of MessageID's for Logged events ( As per 
>>>> Intel design) but not sure that may become difficult when we expand 
>>>> supported ResourceTypes.
>>>
>>> If I am getting correctly, Here is the flow which Intel uses.
>>>
>>>  1. Individual repo have to push the logs using sd_journal_send
>>>     which will write to the file(/var/log/redfish) by using rsyslog
>>>     daemon
>>>
>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>                  "REDFISH_MESSAGE_ID=%s",
>>>                  "ResourceEvent.1.0.ResourceCreated",NULL);
>>>
>>>       * How you would populate the "OriginOfCondition" during
>>>         sending of event?
>>>         (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>
>> Currently in logServices( logEntry),  we are not reporting the 
>> "OriginOfCondition" property as per schema. I will check with Jason( 
>> Who wrote the logService) and get back on this.
>>
>> BTW can you give me how this information is fetched in IBM way of 
>> LogService implementation( D-Bus)? If you already ratified any such, 
>> i think we can leverage.  If not, We work together for solution.
>>
>>>       * Any plan to add resource path in this journal message which
>>>         tells that this is the path for which resource created event
>>>         generated.
>>>
>> Same as above.
>>>
>>>       * Would the path be "Redfish Path/ D-bus Path"?
>>>
>> As per Redfish specification, This should be "@odata.id" which means 
>> it should be of resource uri and so we can't use d-bus path here for 
>> OriginOfConditions.
>>>
>>>       * Where the mapping would be done between D-busPath/Redfish
>>>         Resource path?
>>>
>>>      
>>>           Cons: Every application have to make the change(use sd_journal_send).
>>>           My thought is backend application should not be aware of the redfish terminlogy.
>>
>> Having separate process only for this mapping may not be good( No 
>> different from maintaining that map inside bmcweb as there won't be 
>> any other consumers). Ideal way is, that should be mapped while 
>> logging logEntry's itself. But we are not doing it currently which 
>> need to be re-looked. Give me some time, I will think and check with 
>> other folks and get back.
>>
>>> *2.* Some application(bmcweb) would do the Inotify on the 
>>> path(/var/log/redfish) and send the event once there is any activity 
>>> on this file.
>>>
>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
>>> per Intel design)
>>>
>>>      Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>> Initially i thought "ResourceType" based event filtering can be done 
>> using minimal mapping( Using MessageID and args). But that will work 
>> for minimal set. If the supported ResourceTypes grows, we will have 
>> bigger challenges which i can sense it now. Anyway, Supported 
>> Resources are completely implementation specific. If this value is 
>> empty means, by default all event logs will be sent to subscribers. 
>> This is what we can start with before supported  ResourceTypes list 
>> grows.
>>>>
>>>> As per my reading from below query, You are looking at d-bus match 
>>>> rules and ResourceTypes mapping which is more specific to d-bus 
>>>> event logging(IBM way of implementing event logging). reading it 
>>>> from journal logs will give more information but that will impact 
>>>> the performance to large extent. This might be one of the reason 
>>>> why we (Intel) uses Redfish message ID while logging redfish events 
>>>> logs to journal(You can refer design document for same at 
>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
>>>> In opinion, in your d-bus if you are using some kind of 
>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs 
>>>> for all events and figure out the way to monitor the journal logs 
>>>> without impacting the performance, that should be ok as long as 
>>>> match filters are satisfied for Redfish EventService subscriptions 
>>>> and supported Types(Again differs with implementation).
>>>>
>>>> Thanks,
>>>>
>>>> -Appu
>>>>
>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>> ApparaRao.
>>>>>
>>>>> As you have shown interest in this feature and submitted the 
>>>>> design document, do you have any opinion on this? Do you see any 
>>>>> merit in using D-Bus match in bmcweb to create event logs for life 
>>>>> cycle events?  Please feel free to weigh in.
>>>>>
>>>>> Thanks,
>>>>> Rajes
>>>>>
>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I am going through the bmcweb code for implementing Redfish 
>>>>>> EventService based on the design document 
>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This 
>>>>>> design is hooked to the journal based Redfish Event Logging. For 
>>>>>> life cycle events(ResourceAdded, ResourceRemoved, 
>>>>>> ResourceUpdated),  using D-Bus match, bmcweb can create an event 
>>>>>> log. This requires a JSON dictionary, comprising an array of 
>>>>>> Redfish Resource Name and the D-Bus path. This approach works 
>>>>>> only in case of one to one mapping of Redfish Resource Name and 
>>>>>> the D-Bus path. For propertiesChanged events, if the Redfish 
>>>>>> Resource property is not on the same D-Bus path or the Redfish 
>>>>>> Resource property name is different from the D-Bus property name, 
>>>>>> then an additional JSON dictionary to maintain this information 
>>>>>> is required. With D-Bus match alone in the bmcweb, Redfish 
>>>>>> EventService can't be fully supported. For the Message Registers 
>>>>>> and the Resource Types that are supported, the relevant OpenBMC 
>>>>>> application must create an event log in the journal using either 
>>>>>> the phosphor::logging::entry or sd_journal_send() command.
>>>>>>
>>>>>> After realizing that with D-Bus match in the bmcweb alone can't 
>>>>>> help to fully implement EventService, I prefer to avoid using 
>>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC 
>>>>>> application that generated the event to create an event log in 
>>>>>> the journal. Do you see any advantage of using combination of 
>>>>>> D-Bus match in the bmcweb wherever it is possible and changes to 
>>>>>> OpenBMC application in other cases to create an event log ?
>>>>>>
>>>>>> Your views are highly appreciated.
>>>>>>
>>>>>> Thanks,
>>>>>> Rajes
>>>>>
>>> Thanks
>>> Ratan
>>>
>>>

[-- Attachment #2: Type: text/html, Size: 15338 bytes --]

Re: Redfish EventService Implementation

[RAJESWARAN THILLAIGOVINDAN]

[-- Attachment #1: Type: text/plain, Size: 10624 bytes --]

Apparao,

Thanks a lot for your suggestions. We lean towards using a dictionary to 
map Redfish ResourceType to D-Bus objects path and vice versa and then 
using D-Bus match to generate life cycle events. This way, the changes 
are limited to bmcweb. The resource type and the origin resource URI 
will be included in the event log. This requires change in the format of 
event log file /var/log/redfish. I have commented the same in the server 
sent event patch that you have uploaded. Kindly see if you can leave the 
parsing of file to the OEMs. That way, the existing infrastructure can 
be used by the OEMs to support other filtering mechanisms as defined in 
the specification.

Thanks,
Rajes


On 27-05-2020 09:18, Puli, Apparao wrote:
>
> Hi Rajeswaran,
>
>   Thanks for your mail. At the moment, I don't have plans to support 
> the "ResourceTypes", "OriginResources" based filtering. Basically 
> Intel uses file systems based redfish event logs( journalctl -> 
> rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses. So 
> I am not much familiar with D-Bus logging but some of the suggestions:
>
>  1) While logging redfish events over D-Bus itself,  it can provide 
> details on ResourceTypes and OriginResource URI/Path.
>
>      This is ideal and most efficient way. Since  we walked a walked 
> long distance from start, Its hard to modify all the services to uses 
> these 2 new input parameters while logging events( Requires change in 
> almost all repo's)
>
> 2) For resourcesTypes: Can have mapping dictionary against all 
> MessageId's. For OriginResources: I believe, event log over D-Bus is 
> already holding the Path. If so, last 3/4 nodes of uri can be taken 
> and mapped against the resources and that can be used in Event 
> filtering. We did used same mechanism in case of telemetry  while 
> mapping MetricReportDefinitions to URI.
>
> Hope this helps.
>
> Thanks,
>
> -Appu
>
>
> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>
>> Apparao,
>>
>> I see that you have uploaded a 
>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) 
>> for supporting server sent events. This patch supports event 
>> filtering based on registry prefix  and/or messageId.
>>
>> I would like to know if you have any plan to support event filtering 
>> based on resource type. If so, I would like to work together for a 
>> better solution. Earlier, I have proposed a solution based on D-Bus 
>> match using a dictionary. With that approach, the major challenge is 
>> to map Redfish resource and properties to D-Bus object and properties 
>> respectively.   If D-Bus applications are modified to include 
>> resource type and origin of condition in the event, then there is no 
>> need for a map. But,that brings Redfish terminology to the 
>> application. Also, this will become an overhead if an OEM is not 
>> interested in Redfish event service.
>>
>> Thanks,
>> Rajes
>> On 25-02-2020 19:36, Puli, Apparao wrote:
>>>
>>> Hi Ratan,
>>>
>>>    Comments in-line
>>>
>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>>
>>>> Hi Apparao,
>>>>
>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>>> Hi,
>>>>>
>>>>>   I am sorry for late response as this mail is buried under and 
>>>>> got struck back of my mind.
>>>>>
>>>>> As i did mentioned in EventService design document, EventLog 
>>>>> Monitor service is not common across the organizations( Example: 
>>>>> Intel uses the redfish event logs file and inotify mechanism for 
>>>>> monitoring the event logs. Where as IBM uses d-bus event log 
>>>>> mechanism and they can relay on match rules). That said challenges 
>>>>> with ResourceType mapping will be different in both monitoring 
>>>>> mechanisms. This is good point. Initially when i started 
>>>>> EventService design, i thought we can have mapping in bmcweb for 
>>>>> ResourceTypes with set of MessageID's for Logged events ( As per 
>>>>> Intel design) but not sure that may become difficult when we 
>>>>> expand supported ResourceTypes.
>>>>
>>>> If I am getting correctly, Here is the flow which Intel uses.
>>>>
>>>>  1. Individual repo have to push the logs using sd_journal_send
>>>>     which will write to the file(/var/log/redfish) by using rsyslog
>>>>     daemon
>>>>
>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>>                  "REDFISH_MESSAGE_ID=%s",
>>>>                  "ResourceEvent.1.0.ResourceCreated",NULL);
>>>>
>>>>       * How you would populate the "OriginOfCondition" during
>>>>         sending of event?
>>>>         (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>>
>>> Currently in logServices( logEntry),  we are not reporting the 
>>> "OriginOfCondition" property as per schema. I will check with Jason( 
>>> Who wrote the logService) and get back on this.
>>>
>>> BTW can you give me how this information is fetched in IBM way of 
>>> LogService implementation( D-Bus)? If you already ratified any such, 
>>> i think we can leverage.  If not, We work together for solution.
>>>
>>>>       * Any plan to add resource path in this journal message which
>>>>         tells that this is the path for which resource created
>>>>         event generated.
>>>>
>>> Same as above.
>>>>
>>>>       * Would the path be "Redfish Path/ D-bus Path"?
>>>>
>>> As per Redfish specification, This should be "@odata.id" which means 
>>> it should be of resource uri and so we can't use d-bus path here for 
>>> OriginOfConditions.
>>>>
>>>>       * Where the mapping would be done between D-busPath/Redfish
>>>>         Resource path?
>>>>
>>>>      
>>>>           Cons: Every application have to make the change(use sd_journal_send).
>>>>           My thought is backend application should not be aware of the redfish terminlogy.
>>>
>>> Having separate process only for this mapping may not be good( No 
>>> different from maintaining that map inside bmcweb as there won't be 
>>> any other consumers). Ideal way is, that should be mapped while 
>>> logging logEntry's itself. But we are not doing it currently which 
>>> need to be re-looked. Give me some time, I will think and check with 
>>> other folks and get back.
>>>
>>>> *2.* Some application(bmcweb) would do the Inotify on the 
>>>> path(/var/log/redfish) and send the event once there is any 
>>>> activity on this file.
>>>>
>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
>>>> per Intel design)
>>>>
>>>>      Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>>> Initially i thought "ResourceType" based event filtering can be done 
>>> using minimal mapping( Using MessageID and args). But that will work 
>>> for minimal set. If the supported ResourceTypes grows, we will have 
>>> bigger challenges which i can sense it now.  Anyway, Supported 
>>> Resources are completely implementation specific. If this value is 
>>> empty means, by default all event logs will be sent to subscribers. 
>>> This is what we can start with before supported  ResourceTypes list 
>>> grows.
>>>>>
>>>>> As per my reading from below query, You are looking at d-bus match 
>>>>> rules and ResourceTypes mapping which is more specific to d-bus 
>>>>> event logging(IBM way of implementing event logging). reading it 
>>>>> from journal logs will give more information but that will impact 
>>>>> the performance to large extent. This might be one of the reason 
>>>>> why we (Intel) uses Redfish message ID while logging redfish 
>>>>> events logs to journal(You can refer design document for same at 
>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
>>>>> In opinion, in your d-bus if you are using some kind of 
>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs 
>>>>> for all events and figure out the way to monitor the journal logs 
>>>>> without impacting the performance, that should be ok as long as 
>>>>> match filters are satisfied for Redfish EventService subscriptions 
>>>>> and supported Types(Again differs with implementation).
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Appu
>>>>>
>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>> ApparaRao.
>>>>>>
>>>>>> As you have shown interest in this feature and submitted the 
>>>>>> design document, do you have any opinion on this? Do you see any 
>>>>>> merit in using D-Bus match in bmcweb to create event logs for 
>>>>>> life cycle events?  Please feel free to weigh in.
>>>>>>
>>>>>> Thanks,
>>>>>> Rajes
>>>>>>
>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am going through the bmcweb code for implementing Redfish 
>>>>>>> EventService based on the design document 
>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This 
>>>>>>> design is hooked to the journal based Redfish Event Logging. For 
>>>>>>> life cycle events(ResourceAdded, ResourceRemoved, 
>>>>>>> ResourceUpdated),  using D-Bus match, bmcweb can create an event 
>>>>>>> log. This requires a JSON dictionary, comprising an array of 
>>>>>>> Redfish Resource Name and the D-Bus path. This approach works 
>>>>>>> only in case of one to one mapping of Redfish Resource Name and 
>>>>>>> the D-Bus path. For propertiesChanged events, if the Redfish 
>>>>>>> Resource property is not on the same D-Bus path or the Redfish 
>>>>>>> Resource property name is different from the D-Bus property 
>>>>>>> name, then an additional JSON dictionary to maintain this 
>>>>>>> information is required. With D-Bus match alone in the bmcweb, 
>>>>>>> Redfish EventService can't be fully supported. For the Message 
>>>>>>> Registers and the Resource Types that are supported, the 
>>>>>>> relevant OpenBMC application must create an event log in the 
>>>>>>> journal using either the phosphor::logging::entry or 
>>>>>>> sd_journal_send() command.
>>>>>>>
>>>>>>> After realizing that with D-Bus match in the bmcweb alone can't 
>>>>>>> help to fully implement EventService, I prefer to avoid using 
>>>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC 
>>>>>>> application that generated the event to create an event log in 
>>>>>>> the journal. Do you see any advantage of using combination of 
>>>>>>> D-Bus match in the bmcweb wherever it is possible and changes to 
>>>>>>> OpenBMC application in other cases to create an event log ?
>>>>>>>
>>>>>>> Your views are highly appreciated.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Rajes
>>>>>>
>>>> Thanks
>>>> Ratan
>>>>
>>>>

[-- Attachment #2: Type: text/html, Size: 17042 bytes --]

Re: Redfish EventService Implementation

[Puli, Apparao]

[-- Attachment #1: Type: text/plain, Size: 11603 bytes --]

Rajes,

    The dictionary to map Redfish resourceType and D-Bus object path( I 
believe URI intern) is good. It will be great if you share design 
document, if its done.

At the moment, redfish event logs file(/var/log/redfish) doesn't have 
ResourceTypes and OriginResource fields. The Existing redfish event log 
implementation(log service) also doesn't have support for that. You can 
propose design change for same along with how it is used in event log 
service. Same thing, can be adopted to EventService, once its agreed by 
OEM's( I am thinking, it should go under new OEM specific compiler flag. 
But that we can ratify later).

Thanks,

-Appu

On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>
> Apparao,
>
> Thanks a lot for your suggestions. We lean towards using a dictionary 
> to map Redfish ResourceType to D-Bus objects path and vice versa and 
> then using D-Bus match to generate life cycle events. This way, the 
> changes are limited to bmcweb. The resource type and the origin 
> resource URI will be included in the event log. This requires change 
> in the format of event log file /var/log/redfish. I have commented the 
> same in the server sent event patch that you have uploaded. Kindly see 
> if you can leave the parsing of file to the OEMs. That way, the 
> existing infrastructure can be used by the OEMs to support other 
> filtering mechanisms as defined in the specification.
>
> Thanks,
> Rajes
>
>
> On 27-05-2020 09:18, Puli, Apparao wrote:
>>
>> Hi Rajeswaran,
>>
>>   Thanks for your mail. At the moment, I don't have plans to support 
>> the "ResourceTypes", "OriginResources" based filtering.  Basically 
>> Intel uses file systems based redfish event logs( journalctl -> 
>> rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses. So 
>> I am not much familiar with D-Bus logging but some of the suggestions:
>>
>>  1) While logging redfish events over D-Bus itself,  it can provide 
>> details on ResourceTypes and OriginResource URI/Path.
>>
>>      This is ideal and most efficient way. Since  we walked a walked 
>> long distance from start, Its hard to modify all the services to uses 
>> these 2 new input parameters while logging events( Requires change in 
>> almost all repo's)
>>
>> 2) For resourcesTypes: Can have mapping dictionary against all 
>> MessageId's. For OriginResources: I believe, event log over D-Bus is 
>> already holding the Path. If so, last 3/4 nodes of uri can be taken 
>> and mapped against the resources and that can be used in Event 
>> filtering. We did used same mechanism in case of telemetry  while 
>> mapping MetricReportDefinitions to URI.
>>
>> Hope this helps.
>>
>> Thanks,
>>
>> -Appu
>>
>>
>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>
>>> Apparao,
>>>
>>> I see that you have uploaded a 
>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) 
>>> for supporting server sent events. This patch supports event 
>>> filtering based on registry prefix  and/or messageId.
>>>
>>> I would like to know if you have any plan to support event filtering 
>>> based on resource type. If so, I would like to work together for a 
>>> better solution. Earlier, I have proposed a solution based on D-Bus 
>>> match using a dictionary. With that approach, the major challenge is 
>>> to map Redfish resource and properties to D-Bus object and 
>>> properties respectively.   If D-Bus applications are modified to 
>>> include resource type and origin of condition in the event, then 
>>> there is no need for a map. But,that brings Redfish terminology to 
>>> the application. Also, this will become an overhead if an OEM is not 
>>> interested in Redfish event service.
>>>
>>> Thanks,
>>> Rajes
>>> On 25-02-2020 19:36, Puli, Apparao wrote:
>>>>
>>>> Hi Ratan,
>>>>
>>>>    Comments in-line
>>>>
>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>>>
>>>>> Hi Apparao,
>>>>>
>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>>>> Hi,
>>>>>>
>>>>>>   I am sorry for late response as this mail is buried under and 
>>>>>> got struck back of my mind.
>>>>>>
>>>>>> As i did mentioned in EventService design document, EventLog 
>>>>>> Monitor service is not common across the organizations( Example: 
>>>>>> Intel uses the redfish event logs file and inotify mechanism for 
>>>>>> monitoring the event logs. Where as IBM uses d-bus event log 
>>>>>> mechanism and they can relay on match rules). That said 
>>>>>> challenges with ResourceType mapping will be different in both 
>>>>>> monitoring mechanisms. This is good point. Initially when i 
>>>>>> started EventService design, i thought we can have mapping in 
>>>>>> bmcweb for ResourceTypes with set of MessageID's for Logged 
>>>>>> events ( As per Intel design) but not sure that may become 
>>>>>> difficult when we expand supported ResourceTypes.
>>>>>
>>>>> If I am getting correctly, Here is the flow which Intel uses.
>>>>>
>>>>>  1. Individual repo have to push the logs using sd_journal_send
>>>>>     which will write to the file(/var/log/redfish) by using
>>>>>     rsyslog daemon
>>>>>
>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>>>                  "REDFISH_MESSAGE_ID=%s",
>>>>>                  "ResourceEvent.1.0.ResourceCreated",NULL);
>>>>>
>>>>>       * How you would populate the "OriginOfCondition" during
>>>>>         sending of event?
>>>>>         (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>>>
>>>> Currently in logServices( logEntry),  we are not reporting the 
>>>> "OriginOfCondition" property as per schema. I will check with 
>>>> Jason( Who wrote the logService) and get back on this.
>>>>
>>>> BTW can you give me how this information is fetched in IBM way of 
>>>> LogService implementation( D-Bus)? If you already ratified any 
>>>> such, i think we can leverage.  If not, We work together for solution.
>>>>
>>>>>       * Any plan to add resource path in this journal message
>>>>>         which tells that this is the path for which resource
>>>>>         created event generated.
>>>>>
>>>> Same as above.
>>>>>
>>>>>       * Would the path be "Redfish Path/ D-bus Path"?
>>>>>
>>>> As per Redfish specification, This should be "@odata.id" which 
>>>> means it should be of resource uri and so we can't use d-bus path 
>>>> here for OriginOfConditions.
>>>>>
>>>>>       * Where the mapping would be done between D-busPath/Redfish
>>>>>         Resource path?
>>>>>
>>>>>      
>>>>>           Cons: Every application have to make the change(use sd_journal_send).
>>>>>           My thought is backend application should not be aware of the redfish terminlogy.
>>>>
>>>> Having separate process only for this mapping may not be good( No 
>>>> different from maintaining that map inside bmcweb as there won't be 
>>>> any other consumers). Ideal way is, that should be mapped while 
>>>> logging logEntry's itself. But we are not doing it currently which 
>>>> need to be re-looked. Give me some time, I will think and check 
>>>> with other folks and get back.
>>>>
>>>>> *2.* Some application(bmcweb) would do the Inotify on the 
>>>>> path(/var/log/redfish) and send the event once there is any 
>>>>> activity on this file.
>>>>>
>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
>>>>> per Intel design)
>>>>>
>>>>>      Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>>>> Initially i thought "ResourceType" based event filtering can be 
>>>> done using minimal mapping( Using MessageID and args). But that 
>>>> will work for minimal set. If the supported ResourceTypes grows, we 
>>>> will have bigger challenges which i can sense it now.  Anyway, 
>>>> Supported Resources are completely implementation specific. If this 
>>>> value is empty means, by default all event logs will be sent to 
>>>> subscribers. This is what we can start with before supported  
>>>> ResourceTypes list grows.
>>>>>>
>>>>>> As per my reading from below query, You are looking at d-bus 
>>>>>> match rules and ResourceTypes mapping which is more specific to 
>>>>>> d-bus event logging(IBM way of implementing event logging). 
>>>>>> reading it from journal logs will give more information but that 
>>>>>> will impact the performance to large extent. This might be one of 
>>>>>> the reason why we (Intel) uses Redfish message ID while logging 
>>>>>> redfish events logs to journal(You can refer design document for 
>>>>>> same at 
>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
>>>>>> In opinion, in your d-bus if you are using some kind of 
>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs 
>>>>>> for all events and figure out the way to monitor the journal logs 
>>>>>> without impacting the performance, that should be ok as long as 
>>>>>> match filters are satisfied for Redfish EventService 
>>>>>> subscriptions and supported Types(Again differs with 
>>>>>> implementation).
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> -Appu
>>>>>>
>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>> ApparaRao.
>>>>>>>
>>>>>>> As you have shown interest in this feature and submitted the 
>>>>>>> design document, do you have any opinion on this? Do you see any 
>>>>>>> merit in using D-Bus match in bmcweb to create event logs for 
>>>>>>> life cycle events? Please feel free to weigh in.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Rajes
>>>>>>>
>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I am going through the bmcweb code for implementing Redfish 
>>>>>>>> EventService based on the design document 
>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This 
>>>>>>>> design is hooked to the journal based Redfish Event Logging. 
>>>>>>>> For life cycle events(ResourceAdded, ResourceRemoved, 
>>>>>>>> ResourceUpdated),  using D-Bus match, bmcweb can create an 
>>>>>>>> event log. This requires a JSON dictionary, comprising an array 
>>>>>>>> of Redfish Resource Name and the D-Bus path. This approach 
>>>>>>>> works only in case of one to one mapping of Redfish Resource 
>>>>>>>> Name and the D-Bus path. For propertiesChanged events, if the 
>>>>>>>> Redfish Resource property is not on the same D-Bus path or the 
>>>>>>>> Redfish Resource property name is different from the D-Bus 
>>>>>>>> property name, then an additional JSON dictionary to maintain 
>>>>>>>> this information is required. With D-Bus match alone in the 
>>>>>>>> bmcweb, Redfish EventService can't be fully supported. For the 
>>>>>>>> Message Registers and the Resource Types that are supported, 
>>>>>>>> the relevant OpenBMC application must create an event log in 
>>>>>>>> the journal using either the phosphor::logging::entry or 
>>>>>>>> sd_journal_send() command.
>>>>>>>>
>>>>>>>> After realizing that with D-Bus match in the bmcweb alone can't 
>>>>>>>> help to fully implement EventService, I prefer to avoid using 
>>>>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC 
>>>>>>>> application that generated the event to create an event log in 
>>>>>>>> the journal. Do you see any advantage of using combination of 
>>>>>>>> D-Bus match in the bmcweb wherever it is possible and changes 
>>>>>>>> to OpenBMC application in other cases to create an event log ?
>>>>>>>>
>>>>>>>> Your views are highly appreciated.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Rajes
>>>>>>>
>>>>> Thanks
>>>>> Ratan
>>>>>
>>>>>

[-- Attachment #2: Type: text/html, Size: 18852 bytes --]

Re: Redfish EventService Implementation

[Ratan Gupta]

[-- Attachment #1: Type: text/plain, Size: 12160 bytes --]

Hi Appu,

Can you get me an example say if I want an redfish event to be generated 
once any local user gets added on the BMC.

What would be the steps to be done with the current design and 
where(which repo)?


On 5/28/20 12:28 AM, Puli, Apparao wrote:
>
> Rajes,
>
>    The dictionary to map Redfish resourceType and D-Bus object path( I 
> believe URI intern) is good. It will be great if you share design 
> document, if its done.
>
> At the moment, redfish event logs file(/var/log/redfish) doesn't have 
> ResourceTypes and OriginResource fields. The Existing redfish event 
> log implementation(log service) also doesn't have support for that. 
> You can propose design change for same along with how it is used in 
> event log service. Same thing, can be adopted to EventService, once 
> its agreed by OEM's( I am thinking, it should go under new OEM 
> specific compiler flag. But that we can ratify later).
>
> Thanks,
>
> -Appu
>
> On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>
>> Apparao,
>>
>> Thanks a lot for your suggestions. We lean towards using a dictionary 
>> to map Redfish ResourceType to D-Bus objects path and vice versa and 
>> then using D-Bus match to generate life cycle events. This way, the 
>> changes are limited to bmcweb. The resource type and the origin 
>> resource URI will be included in the event log. This requires change 
>> in the format of event log file /var/log/redfish. I have commented 
>> the same in the server sent event patch that you have uploaded. 
>> Kindly see if you can leave the parsing of file to the OEMs. That 
>> way, the existing infrastructure can be used by the OEMs to support 
>> other filtering mechanisms as defined in the specification.
>>
>> Thanks,
>> Rajes
>>
>>
>> On 27-05-2020 09:18, Puli, Apparao wrote:
>>>
>>> Hi Rajeswaran,
>>>
>>>   Thanks for your mail. At the moment, I don't have plans to support 
>>> the "ResourceTypes", "OriginResources" based filtering.  Basically 
>>> Intel uses file systems based redfish event logs( journalctl -> 
>>> rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses. 
>>> So I am not much familiar with D-Bus logging but some of the 
>>> suggestions:
>>>
>>>  1) While logging redfish events over D-Bus itself,  it can provide 
>>> details on ResourceTypes and OriginResource URI/Path.
>>>
>>>      This is ideal and most efficient way. Since  we walked a walked 
>>> long distance from start, Its hard to modify all the services to 
>>> uses these 2 new input parameters while logging events( Requires 
>>> change in almost all repo's)
>>>
>>> 2) For resourcesTypes: Can have mapping dictionary against all 
>>> MessageId's. For OriginResources: I believe, event log over D-Bus is 
>>> already holding the Path. If so, last 3/4 nodes of uri can be taken 
>>> and mapped against the resources and that can be used in Event 
>>> filtering. We did used same mechanism in case of telemetry  while 
>>> mapping MetricReportDefinitions to URI.
>>>
>>> Hope this helps.
>>>
>>> Thanks,
>>>
>>> -Appu
>>>
>>>
>>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>
>>>> Apparao,
>>>>
>>>> I see that you have uploaded a 
>>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) 
>>>> for supporting server sent events. This patch supports event 
>>>> filtering based on registry prefix  and/or messageId.
>>>>
>>>> I would like to know if you have any plan to support event 
>>>> filtering based on resource type. If so, I would like to work 
>>>> together for a better solution. Earlier, I have proposed a solution 
>>>> based on D-Bus match using a dictionary. With that approach, the 
>>>> major challenge is to map Redfish resource and properties to D-Bus 
>>>> object and properties respectively.   If D-Bus applications are 
>>>> modified to include resource type and origin of condition in the 
>>>> event, then there is no need for a map. But,that brings Redfish 
>>>> terminology to the application. Also, this will become an overhead 
>>>> if an OEM is not interested in Redfish event service.
>>>>
>>>> Thanks,
>>>> Rajes
>>>> On 25-02-2020 19:36, Puli, Apparao wrote:
>>>>>
>>>>> Hi Ratan,
>>>>>
>>>>>    Comments in-line
>>>>>
>>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>>>>
>>>>>> Hi Apparao,
>>>>>>
>>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>>   I am sorry for late response as this mail is buried under and 
>>>>>>> got struck back of my mind.
>>>>>>>
>>>>>>> As i did mentioned in EventService design document, EventLog 
>>>>>>> Monitor service is not common across the organizations( Example: 
>>>>>>> Intel uses the redfish event logs file and inotify mechanism for 
>>>>>>> monitoring the event logs. Where as IBM uses d-bus event log 
>>>>>>> mechanism and they can relay on match rules). That said 
>>>>>>> challenges with ResourceType mapping will be different in both 
>>>>>>> monitoring mechanisms. This is good point. Initially when i 
>>>>>>> started EventService design, i thought we can have mapping in 
>>>>>>> bmcweb for ResourceTypes with set of MessageID's for Logged 
>>>>>>> events ( As per Intel design) but not sure that may become 
>>>>>>> difficult when we expand supported ResourceTypes.
>>>>>>
>>>>>> If I am getting correctly, Here is the flow which Intel uses.
>>>>>>
>>>>>>  1. Individual repo have to push the logs using sd_journal_send
>>>>>>     which will write to the file(/var/log/redfish) by using
>>>>>>     rsyslog daemon
>>>>>>
>>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>>>>                  "REDFISH_MESSAGE_ID=%s",
>>>>>>                  "ResourceEvent.1.0.ResourceCreated",NULL);
>>>>>>
>>>>>>       * How you would populate the "OriginOfCondition" during
>>>>>>         sending of event?
>>>>>>         (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>>>>
>>>>> Currently in logServices( logEntry),  we are not reporting the 
>>>>> "OriginOfCondition" property as per schema. I will check with 
>>>>> Jason( Who wrote the logService) and get back on this.
>>>>>
>>>>> BTW can you give me how this information is fetched in IBM way of 
>>>>> LogService implementation( D-Bus)? If you already ratified any 
>>>>> such, i think we can leverage.  If not, We work together for 
>>>>> solution.
>>>>>
>>>>>>       * Any plan to add resource path in this journal message
>>>>>>         which tells that this is the path for which resource
>>>>>>         created event generated.
>>>>>>
>>>>> Same as above.
>>>>>>
>>>>>>       * Would the path be "Redfish Path/ D-bus Path"?
>>>>>>
>>>>> As per Redfish specification, This should be "@odata.id" which 
>>>>> means it should be of resource uri and so we can't use d-bus path 
>>>>> here for OriginOfConditions.
>>>>>>
>>>>>>       * Where the mapping would be done between D-busPath/Redfish
>>>>>>         Resource path?
>>>>>>
>>>>>>      
>>>>>>           Cons: Every application have to make the change(use sd_journal_send).
>>>>>>           My thought is backend application should not be aware of the redfish terminlogy.
>>>>>
>>>>> Having separate process only for this mapping may not be good( No 
>>>>> different from maintaining that map inside bmcweb as there won't 
>>>>> be any other consumers). Ideal way is, that should be mapped while 
>>>>> logging logEntry's itself. But we are not doing it currently which 
>>>>> need to be re-looked. Give me some time, I will think and check 
>>>>> with other folks and get back.
>>>>>
>>>>>> *2.* Some application(bmcweb) would do the Inotify on the 
>>>>>> path(/var/log/redfish) and send the event once there is any 
>>>>>> activity on this file.
>>>>>>
>>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
>>>>>> per Intel design)
>>>>>>
>>>>>>      Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>>>>> Initially i thought "ResourceType" based event filtering can be 
>>>>> done using minimal mapping( Using MessageID and args). But that 
>>>>> will work for minimal set. If the supported ResourceTypes grows, 
>>>>> we will have bigger challenges which i can sense it now.  Anyway, 
>>>>> Supported Resources are completely implementation specific. If 
>>>>> this value is empty means, by default all event logs will be sent 
>>>>> to subscribers. This is what we can start with before supported  
>>>>> ResourceTypes list grows.
>>>>>>>
>>>>>>> As per my reading from below query, You are looking at d-bus 
>>>>>>> match rules and ResourceTypes mapping which is more specific to 
>>>>>>> d-bus event logging(IBM way of implementing event logging). 
>>>>>>> reading it from journal logs will give more information but that 
>>>>>>> will impact the performance to large extent. This might be one 
>>>>>>> of the reason why we (Intel) uses Redfish message ID while 
>>>>>>> logging redfish events logs to journal(You can refer design 
>>>>>>> document for same at 
>>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
>>>>>>> In opinion, in your d-bus if you are using some kind of 
>>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs 
>>>>>>> for all events and figure out the way to monitor the journal 
>>>>>>> logs without impacting the performance, that should be ok as 
>>>>>>> long as match filters are satisfied for Redfish EventService 
>>>>>>> subscriptions and supported Types(Again differs with 
>>>>>>> implementation).
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> -Appu
>>>>>>>
>>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>> ApparaRao.
>>>>>>>>
>>>>>>>> As you have shown interest in this feature and submitted the 
>>>>>>>> design document, do you have any opinion on this? Do you see 
>>>>>>>> any merit in using D-Bus match in bmcweb to create event logs 
>>>>>>>> for life cycle events?  Please feel free to weigh in.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Rajes
>>>>>>>>
>>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I am going through the bmcweb code for implementing Redfish 
>>>>>>>>> EventService based on the design document 
>>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. 
>>>>>>>>> This design is hooked to the journal based Redfish Event 
>>>>>>>>> Logging. For life cycle events(ResourceAdded, ResourceRemoved, 
>>>>>>>>> ResourceUpdated),  using D-Bus match, bmcweb can create an 
>>>>>>>>> event log. This requires a JSON dictionary, comprising an 
>>>>>>>>> array of Redfish Resource Name and the D-Bus path. This 
>>>>>>>>> approach works only in case of one to one mapping of Redfish 
>>>>>>>>> Resource Name and the D-Bus path. For propertiesChanged 
>>>>>>>>> events, if the Redfish Resource property is not on the same 
>>>>>>>>> D-Bus path or the Redfish Resource property name is different 
>>>>>>>>> from the D-Bus property name, then an additional JSON 
>>>>>>>>> dictionary to maintain this information is required. With 
>>>>>>>>> D-Bus match alone in the bmcweb, Redfish EventService can't be 
>>>>>>>>> fully supported. For the Message Registers and the Resource 
>>>>>>>>> Types that are supported, the relevant OpenBMC application 
>>>>>>>>> must create an event log in the journal using either the 
>>>>>>>>> phosphor::logging::entry or sd_journal_send() command.
>>>>>>>>>
>>>>>>>>> After realizing that with D-Bus match in the bmcweb alone 
>>>>>>>>> can't help to fully implement EventService, I prefer to avoid 
>>>>>>>>> using D-Bus match in bmcweb. Instead, I prefer to modify the 
>>>>>>>>> OpenBMC application that generated the event to create an 
>>>>>>>>> event log in the journal. Do you see any advantage of using 
>>>>>>>>> combination of D-Bus match in the bmcweb wherever it is 
>>>>>>>>> possible and changes to OpenBMC application in other cases to 
>>>>>>>>> create an event log ?
>>>>>>>>>
>>>>>>>>> Your views are highly appreciated.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Rajes
>>>>>>>>
>>>>>> Thanks
>>>>>> Ratan
>>>>>>
>>>>>>
Ratan

[-- Attachment #2: Type: text/html, Size: 20206 bytes --]

Re: Redfish event log for new local user addition

[Puli, Apparao]

[-- Attachment #1: Type: text/plain, Size: 13233 bytes --]

[Subject changed to reflect actual context]

Hi Ratan,

   With current OpenBMC, I don't any event log getting added for new 
local user creation. To implement that below are two place we need to 
add support:

1) Define an new message retry ID for user addition(say UserAdded) in 
bmcweb.

https://github.com/openbmc/bmcweb/blob/master/redfish-core/include/registries/openbmc_message_registry.hpp

2) Add a event log during new user addition inside 
phospor-user-manager(createUser function).

https://github.com/openbmc/phosphor-user-manager/blob/master/user_mgr.cpp#L336


While you are adding event log for new local user creation, Please 
consider other case too( Like Delete, Update, Rename etc..)

Thanks,

-Appu


On 5/28/2020 6:56 PM, Ratan Gupta wrote:
>
> Hi Appu,
>
> Can you get me an example say if I want an redfish event to be 
> generated once any local user gets added on the BMC.
>
> What would be the steps to be done with the current design and 
> where(which repo)?
>
>
> On 5/28/20 12:28 AM, Puli, Apparao wrote:
>>
>> Rajes,
>>
>>    The dictionary to map Redfish resourceType and D-Bus object path( 
>> I believe URI intern) is good. It will be great if you share design 
>> document, if its done.
>>
>> At the moment, redfish event logs file(/var/log/redfish) doesn't have 
>> ResourceTypes and OriginResource fields. The Existing redfish event 
>> log implementation(log service) also doesn't have support for that. 
>> You can propose design change for same along with how it is used in 
>> event log service. Same thing, can be adopted to EventService, once 
>> its agreed by OEM's( I am thinking, it should go under new OEM 
>> specific compiler flag. But that we can ratify later).
>>
>> Thanks,
>>
>> -Appu
>>
>> On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>
>>> Apparao,
>>>
>>> Thanks a lot for your suggestions. We lean towards using a 
>>> dictionary to map Redfish ResourceType to D-Bus objects path and 
>>> vice versa and then using D-Bus match to generate life cycle events. 
>>> This way, the changes are limited to bmcweb. The resource type and 
>>> the origin resource URI will be included in the event log. This 
>>> requires change in the format of event log file /var/log/redfish. I 
>>> have commented the same in the server sent event patch that you have 
>>> uploaded. Kindly see if you can leave the parsing of file to the 
>>> OEMs. That way, the existing infrastructure can be used by the OEMs 
>>> to support other filtering mechanisms as defined in the specification.
>>>
>>> Thanks,
>>> Rajes
>>>
>>>
>>> On 27-05-2020 09:18, Puli, Apparao wrote:
>>>>
>>>> Hi Rajeswaran,
>>>>
>>>>   Thanks for your mail. At the moment, I don't have plans to 
>>>> support the "ResourceTypes", "OriginResources" based filtering.  
>>>> Basically Intel uses file systems based redfish event logs( 
>>>> journalctl -> rsync-> filesystem) and doesn't use D-Bus mechanism 
>>>> like IBM uses. So I am not much familiar with D-Bus logging but 
>>>> some of the suggestions:
>>>>
>>>>  1) While logging redfish events over D-Bus itself,  it can provide 
>>>> details on ResourceTypes and OriginResource URI/Path.
>>>>
>>>>      This is ideal and most efficient way. Since  we walked a 
>>>> walked long distance from start, Its hard to modify all the 
>>>> services to uses these 2 new input parameters while logging events( 
>>>> Requires change in almost all repo's)
>>>>
>>>> 2) For resourcesTypes: Can have mapping dictionary against all 
>>>> MessageId's. For OriginResources: I believe, event log over D-Bus 
>>>> is already holding the Path. If so, last 3/4 nodes of uri can be 
>>>> taken and mapped against the resources and that can be used in 
>>>> Event filtering. We did used same mechanism in case of telemetry  
>>>> while mapping MetricReportDefinitions to URI.
>>>>
>>>> Hope this helps.
>>>>
>>>> Thanks,
>>>>
>>>> -Appu
>>>>
>>>>
>>>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>
>>>>> Apparao,
>>>>>
>>>>> I see that you have uploaded a 
>>>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) 
>>>>> for supporting server sent events. This patch supports event 
>>>>> filtering based on registry prefix  and/or messageId.
>>>>>
>>>>> I would like to know if you have any plan to support event 
>>>>> filtering based on resource type. If so, I would like to work 
>>>>> together for a better solution. Earlier, I have proposed a 
>>>>> solution based on D-Bus match using a dictionary. With that 
>>>>> approach, the major challenge is to map Redfish resource and 
>>>>> properties to D-Bus object and properties respectively.   If D-Bus 
>>>>> applications are modified to include resource type and origin of 
>>>>> condition in the event, then there is no need for a map. But,that 
>>>>> brings Redfish terminology to the application. Also, this will 
>>>>> become an overhead if an OEM is not interested in Redfish event 
>>>>> service.
>>>>>
>>>>> Thanks,
>>>>> Rajes
>>>>> On 25-02-2020 19:36, Puli, Apparao wrote:
>>>>>>
>>>>>> Hi Ratan,
>>>>>>
>>>>>>    Comments in-line
>>>>>>
>>>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>>>>>
>>>>>>> Hi Apparao,
>>>>>>>
>>>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>>   I am sorry for late response as this mail is buried under and 
>>>>>>>> got struck back of my mind.
>>>>>>>>
>>>>>>>> As i did mentioned in EventService design document, EventLog 
>>>>>>>> Monitor service is not common across the organizations( 
>>>>>>>> Example: Intel uses the redfish event logs file and inotify 
>>>>>>>> mechanism for monitoring the event logs. Where as IBM uses 
>>>>>>>> d-bus event log mechanism and they can relay on match rules). 
>>>>>>>> That said challenges with ResourceType mapping will be 
>>>>>>>> different in both monitoring mechanisms. This is good point. 
>>>>>>>> Initially when i started EventService design, i thought we can 
>>>>>>>> have mapping in bmcweb for ResourceTypes with set of 
>>>>>>>> MessageID's for Logged events ( As per Intel design) but not 
>>>>>>>> sure that may become difficult when we expand supported 
>>>>>>>> ResourceTypes.
>>>>>>>
>>>>>>> If I am getting correctly, Here is the flow which Intel uses.
>>>>>>>
>>>>>>>  1. Individual repo have to push the logs using sd_journal_send
>>>>>>>     which will write to the file(/var/log/redfish) by using
>>>>>>>     rsyslog daemon
>>>>>>>
>>>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>>>>>                  "REDFISH_MESSAGE_ID=%s",
>>>>>>>                  "ResourceEvent.1.0.ResourceCreated",NULL);
>>>>>>>
>>>>>>>       * How you would populate the "OriginOfCondition" during
>>>>>>>         sending of event?
>>>>>>>         (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>>>>>
>>>>>> Currently in logServices( logEntry),  we are not reporting the 
>>>>>> "OriginOfCondition" property as per schema. I will check with 
>>>>>> Jason( Who wrote the logService) and get back on this.
>>>>>>
>>>>>> BTW can you give me how this information is fetched in IBM way of 
>>>>>> LogService implementation( D-Bus)? If you already ratified any 
>>>>>> such, i think we can leverage.  If not, We work together for 
>>>>>> solution.
>>>>>>
>>>>>>>       * Any plan to add resource path in this journal message
>>>>>>>         which tells that this is the path for which resource
>>>>>>>         created event generated.
>>>>>>>
>>>>>> Same as above.
>>>>>>>
>>>>>>>       * Would the path be "Redfish Path/ D-bus Path"?
>>>>>>>
>>>>>> As per Redfish specification, This should be "@odata.id" which 
>>>>>> means it should be of resource uri and so we can't use d-bus path 
>>>>>> here for OriginOfConditions.
>>>>>>>
>>>>>>>       * Where the mapping would be done between
>>>>>>>         D-busPath/Redfish Resource path?
>>>>>>>
>>>>>>>      
>>>>>>>           Cons: Every application have to make the change(use sd_journal_send).
>>>>>>>           My thought is backend application should not be aware of the redfish terminlogy.
>>>>>>
>>>>>> Having separate process only for this mapping may not be good( No 
>>>>>> different from maintaining that map inside bmcweb as there won't 
>>>>>> be any other consumers). Ideal way is, that should be mapped 
>>>>>> while logging logEntry's itself. But we are not doing it 
>>>>>> currently which need to be re-looked. Give me some time, I will 
>>>>>> think and check with other folks and get back.
>>>>>>
>>>>>>> *2.* Some application(bmcweb) would do the Inotify on the 
>>>>>>> path(/var/log/redfish) and send the event once there is any 
>>>>>>> activity on this file.
>>>>>>>
>>>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
>>>>>>> per Intel design)
>>>>>>>
>>>>>>>      Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>>>>>> Initially i thought "ResourceType" based event filtering can be 
>>>>>> done using minimal mapping( Using MessageID and args). But that 
>>>>>> will work for minimal set. If the supported ResourceTypes grows, 
>>>>>> we will have bigger challenges which i can sense it now.  Anyway, 
>>>>>> Supported Resources are completely implementation specific. If 
>>>>>> this value is empty means, by default all event logs will be sent 
>>>>>> to subscribers. This is what we can start with before supported  
>>>>>> ResourceTypes list grows.
>>>>>>>>
>>>>>>>> As per my reading from below query, You are looking at d-bus 
>>>>>>>> match rules and ResourceTypes mapping which is more specific to 
>>>>>>>> d-bus event logging(IBM way of implementing event logging). 
>>>>>>>> reading it from journal logs will give more information but 
>>>>>>>> that will impact the performance to large extent. This might be 
>>>>>>>> one of the reason why we (Intel) uses Redfish message ID while 
>>>>>>>> logging redfish events logs to journal(You can refer design 
>>>>>>>> document for same at 
>>>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
>>>>>>>> In opinion, in your d-bus if you are using some kind of 
>>>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal 
>>>>>>>> logs for all events and figure out the way to monitor the 
>>>>>>>> journal logs without impacting the performance, that should be 
>>>>>>>> ok as long as match filters are satisfied for Redfish 
>>>>>>>> EventService subscriptions and supported Types(Again differs 
>>>>>>>> with implementation).
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> -Appu
>>>>>>>>
>>>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>>> ApparaRao.
>>>>>>>>>
>>>>>>>>> As you have shown interest in this feature and submitted the 
>>>>>>>>> design document, do you have any opinion on this? Do you see 
>>>>>>>>> any merit in using D-Bus match in bmcweb to create event logs 
>>>>>>>>> for life cycle events?  Please feel free to weigh in.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Rajes
>>>>>>>>>
>>>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I am going through the bmcweb code for implementing Redfish 
>>>>>>>>>> EventService based on the design document 
>>>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. 
>>>>>>>>>> This design is hooked to the journal based Redfish Event 
>>>>>>>>>> Logging. For life cycle events(ResourceAdded, 
>>>>>>>>>> ResourceRemoved, ResourceUpdated),  using D-Bus match, bmcweb 
>>>>>>>>>> can create an event log. This requires a JSON dictionary, 
>>>>>>>>>> comprising an array of Redfish Resource Name and the D-Bus 
>>>>>>>>>> path. This approach works only in case of one to one mapping 
>>>>>>>>>> of Redfish Resource Name and the D-Bus path. For 
>>>>>>>>>> propertiesChanged events, if the Redfish Resource property is 
>>>>>>>>>> not on the same D-Bus path or the Redfish Resource property 
>>>>>>>>>> name is different from the D-Bus property name, then an 
>>>>>>>>>> additional JSON dictionary to maintain this information is 
>>>>>>>>>> required. With D-Bus match alone in the bmcweb, Redfish 
>>>>>>>>>> EventService can't be fully supported. For the Message 
>>>>>>>>>> Registers and the Resource Types that are supported, the 
>>>>>>>>>> relevant OpenBMC application must create an event log in the 
>>>>>>>>>> journal using either the phosphor::logging::entry or 
>>>>>>>>>> sd_journal_send() command.
>>>>>>>>>>
>>>>>>>>>> After realizing that with D-Bus match in the bmcweb alone 
>>>>>>>>>> can't help to fully implement EventService, I prefer to avoid 
>>>>>>>>>> using D-Bus match in bmcweb. Instead, I prefer to modify the 
>>>>>>>>>> OpenBMC application that generated the event to create an 
>>>>>>>>>> event log in the journal. Do you see any advantage of using 
>>>>>>>>>> combination of D-Bus match in the bmcweb wherever it is 
>>>>>>>>>> possible and changes to OpenBMC application in other cases to 
>>>>>>>>>> create an event log ?
>>>>>>>>>>
>>>>>>>>>> Your views are highly appreciated.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Rajes
>>>>>>>>>
>>>>>>> Thanks
>>>>>>> Ratan
>>>>>>>
>>>>>>>
> Ratan

[-- Attachment #2: Type: text/html, Size: 22528 bytes --]

Re: Redfish event log for new local user addition

[Ratan Gupta]

[-- Attachment #1: Type: text/plain, Size: 13972 bytes --]

Hi Appu,

Thanks for responding.

As the Impact would be on all the openbmc repo where we need to add the 
event log for any properties change or interface added.

I feel it is kind of overhead and not realistic as each and every repo 
need to know about redfish terminology eg : Redfish Path, Redfish 
property, Redfish Registry Prefix etc.

I feel it is a big work.

Ratan

On 5/29/20 9:15 PM, Puli, Apparao wrote:
>
> [Subject changed to reflect actual context]
>
> Hi Ratan,
>
>   With current OpenBMC, I don't any event log getting added for new 
> local user creation. To implement that below are two place we need to 
> add support:
>
> 1) Define an new message retry ID for user addition(say UserAdded) in 
> bmcweb.
>
> https://github.com/openbmc/bmcweb/blob/master/redfish-core/include/registries/openbmc_message_registry.hpp
>
> 2) Add a event log during new user addition inside 
> phospor-user-manager(createUser function).
>
> https://github.com/openbmc/phosphor-user-manager/blob/master/user_mgr.cpp#L336
>
>
> While you are adding event log for new local user creation, Please 
> consider other case too( Like Delete, Update, Rename etc..)
>
> Thanks,
>
> -Appu
>
>
> On 5/28/2020 6:56 PM, Ratan Gupta wrote:
>>
>> Hi Appu,
>>
>> Can you get me an example say if I want an redfish event to be 
>> generated once any local user gets added on the BMC.
>>
>> What would be the steps to be done with the current design and 
>> where(which repo)?
>>
>>
>> On 5/28/20 12:28 AM, Puli, Apparao wrote:
>>>
>>> Rajes,
>>>
>>>    The dictionary to map Redfish resourceType and D-Bus object path( 
>>> I believe URI intern) is good. It will be great if you share design 
>>> document, if its done.
>>>
>>> At the moment, redfish event logs file(/var/log/redfish) doesn't 
>>> have ResourceTypes and OriginResource fields. The Existing redfish 
>>> event log implementation(log service) also doesn't have support for 
>>> that. You can propose design change for same along with how it is 
>>> used in event log service. Same thing, can be adopted to 
>>> EventService, once its agreed by OEM's( I am thinking, it should go 
>>> under new OEM specific compiler flag. But that we can ratify later).
>>>
>>> Thanks,
>>>
>>> -Appu
>>>
>>> On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>
>>>> Apparao,
>>>>
>>>> Thanks a lot for your suggestions. We lean towards using a 
>>>> dictionary to map Redfish ResourceType to D-Bus objects path and 
>>>> vice versa and then using D-Bus match to generate life cycle 
>>>> events. This way, the changes are limited to bmcweb. The resource 
>>>> type and the origin resource URI will be included in the event log. 
>>>> This requires change in the format of event log file 
>>>> /var/log/redfish. I have commented the same in the server sent 
>>>> event patch that you have uploaded. Kindly see if you can leave the 
>>>> parsing of file to the OEMs. That way, the existing infrastructure 
>>>> can be used by the OEMs to support other filtering mechanisms as 
>>>> defined in the specification.
>>>>
>>>> Thanks,
>>>> Rajes
>>>>
>>>>
>>>> On 27-05-2020 09:18, Puli, Apparao wrote:
>>>>>
>>>>> Hi Rajeswaran,
>>>>>
>>>>>   Thanks for your mail. At the moment, I don't have plans to 
>>>>> support the "ResourceTypes", "OriginResources" based filtering.  
>>>>> Basically Intel uses file systems based redfish event logs( 
>>>>> journalctl -> rsync-> filesystem) and doesn't use D-Bus mechanism 
>>>>> like IBM uses. So I am not much familiar with D-Bus logging but 
>>>>> some of the suggestions:
>>>>>
>>>>>  1) While logging redfish events over D-Bus itself,  it can 
>>>>> provide details on ResourceTypes and OriginResource URI/Path.
>>>>>
>>>>>      This is ideal and most efficient way. Since  we walked a 
>>>>> walked long distance from start, Its hard to modify all the 
>>>>> services to uses these 2 new input parameters while logging 
>>>>> events( Requires change in almost all repo's)
>>>>>
>>>>> 2) For resourcesTypes: Can have mapping dictionary against all 
>>>>> MessageId's. For OriginResources: I believe, event log over D-Bus 
>>>>> is already holding the Path. If so, last 3/4 nodes of uri can be 
>>>>> taken and mapped against the resources and that can be used in 
>>>>> Event filtering. We did used same mechanism in case of telemetry  
>>>>> while mapping MetricReportDefinitions to URI.
>>>>>
>>>>> Hope this helps.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Appu
>>>>>
>>>>>
>>>>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>
>>>>>> Apparao,
>>>>>>
>>>>>> I see that you have uploaded a 
>>>>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) 
>>>>>> for supporting server sent events. This patch supports event 
>>>>>> filtering based on registry prefix  and/or messageId.
>>>>>>
>>>>>> I would like to know if you have any plan to support event 
>>>>>> filtering based on resource type. If so, I would like to work 
>>>>>> together for a better solution. Earlier, I have proposed a 
>>>>>> solution based on D-Bus match using a dictionary. With that 
>>>>>> approach, the major challenge is to map Redfish resource and 
>>>>>> properties to D-Bus object and properties respectively.   If 
>>>>>> D-Bus applications are modified to include resource type and 
>>>>>> origin of condition in the event, then there is no need for a 
>>>>>> map. But,that brings Redfish terminology to the application. 
>>>>>> Also, this will become an overhead if an OEM is not interested in 
>>>>>> Redfish event service.
>>>>>>
>>>>>> Thanks,
>>>>>> Rajes
>>>>>> On 25-02-2020 19:36, Puli, Apparao wrote:
>>>>>>>
>>>>>>> Hi Ratan,
>>>>>>>
>>>>>>>    Comments in-line
>>>>>>>
>>>>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>>>>>>
>>>>>>>> Hi Apparao,
>>>>>>>>
>>>>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>>   I am sorry for late response as this mail is buried under 
>>>>>>>>> and got struck back of my mind.
>>>>>>>>>
>>>>>>>>> As i did mentioned in EventService design document, EventLog 
>>>>>>>>> Monitor service is not common across the organizations( 
>>>>>>>>> Example: Intel uses the redfish event logs file and inotify 
>>>>>>>>> mechanism for monitoring the event logs. Where as IBM uses 
>>>>>>>>> d-bus event log mechanism and they can relay on match rules). 
>>>>>>>>> That said challenges with ResourceType mapping will be 
>>>>>>>>> different in both monitoring mechanisms. This is good point. 
>>>>>>>>> Initially when i started EventService design, i thought we can 
>>>>>>>>> have mapping in bmcweb for ResourceTypes with set of 
>>>>>>>>> MessageID's for Logged events ( As per Intel design) but not 
>>>>>>>>> sure that may become difficult when we expand supported 
>>>>>>>>> ResourceTypes.
>>>>>>>>
>>>>>>>> If I am getting correctly, Here is the flow which Intel uses.
>>>>>>>>
>>>>>>>>  1. Individual repo have to push the logs using sd_journal_send
>>>>>>>>     which will write to the file(/var/log/redfish) by using
>>>>>>>>     rsyslog daemon
>>>>>>>>
>>>>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>>>>>>                  "REDFISH_MESSAGE_ID=%s",
>>>>>>>>                  "ResourceEvent.1.0.ResourceCreated",NULL);
>>>>>>>>
>>>>>>>>       * How you would populate the "OriginOfCondition" during
>>>>>>>>         sending of event?
>>>>>>>>         (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>>>>>>
>>>>>>> Currently in logServices( logEntry),  we are not reporting the 
>>>>>>> "OriginOfCondition" property as per schema. I will check with 
>>>>>>> Jason( Who wrote the logService) and get back on this.
>>>>>>>
>>>>>>> BTW can you give me how this information is fetched in IBM way 
>>>>>>> of LogService implementation( D-Bus)? If you already ratified 
>>>>>>> any such, i think we can leverage.  If not, We work together for 
>>>>>>> solution.
>>>>>>>
>>>>>>>>       * Any plan to add resource path in this journal message
>>>>>>>>         which tells that this is the path for which resource
>>>>>>>>         created event generated.
>>>>>>>>
>>>>>>> Same as above.
>>>>>>>>
>>>>>>>>       * Would the path be "Redfish Path/ D-bus Path"?
>>>>>>>>
>>>>>>> As per Redfish specification, This should be "@odata.id" which 
>>>>>>> means it should be of resource uri and so we can't use d-bus 
>>>>>>> path here for OriginOfConditions.
>>>>>>>>
>>>>>>>>       * Where the mapping would be done between
>>>>>>>>         D-busPath/Redfish Resource path?
>>>>>>>>
>>>>>>>>      
>>>>>>>>           Cons: Every application have to make the change(use sd_journal_send).
>>>>>>>>           My thought is backend application should not be aware of the redfish terminlogy.
>>>>>>>
>>>>>>> Having separate process only for this mapping may not be good( 
>>>>>>> No different from maintaining that map inside bmcweb as there 
>>>>>>> won't be any other consumers). Ideal way is, that should be 
>>>>>>> mapped while logging logEntry's itself. But we are not doing it 
>>>>>>> currently which need to be re-looked. Give me some time, I will 
>>>>>>> think and check with other folks and get back.
>>>>>>>
>>>>>>>> *2.* Some application(bmcweb) would do the Inotify on the 
>>>>>>>> path(/var/log/redfish) and send the event once there is any 
>>>>>>>> activity on this file.
>>>>>>>>
>>>>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As 
>>>>>>>> per Intel design)
>>>>>>>>
>>>>>>>>      Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>>>>>>> Initially i thought "ResourceType" based event filtering can be 
>>>>>>> done using minimal mapping( Using MessageID and args). But that 
>>>>>>> will work for minimal set. If the supported ResourceTypes grows, 
>>>>>>> we will have bigger challenges which i can sense it now. Anyway, 
>>>>>>> Supported Resources are completely implementation specific. If 
>>>>>>> this value is empty means, by default all event logs will be 
>>>>>>> sent to subscribers. This is what we can start with before 
>>>>>>> supported ResourceTypes list grows.
>>>>>>>>>
>>>>>>>>> As per my reading from below query, You are looking at d-bus 
>>>>>>>>> match rules and ResourceTypes mapping which is more specific 
>>>>>>>>> to d-bus event logging(IBM way of implementing event logging). 
>>>>>>>>> reading it from journal logs will give more information but 
>>>>>>>>> that will impact the performance to large extent. This might 
>>>>>>>>> be one of the reason why we (Intel) uses Redfish message ID 
>>>>>>>>> while logging redfish events logs to journal(You can refer 
>>>>>>>>> design document for same at 
>>>>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). 
>>>>>>>>> In opinion, in your d-bus if you are using some kind of 
>>>>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal 
>>>>>>>>> logs for all events and figure out the way to monitor the 
>>>>>>>>> journal logs without impacting the performance, that should be 
>>>>>>>>> ok as long as match filters are satisfied for Redfish 
>>>>>>>>> EventService subscriptions and supported Types(Again differs 
>>>>>>>>> with implementation).
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> -Appu
>>>>>>>>>
>>>>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>>>> ApparaRao.
>>>>>>>>>>
>>>>>>>>>> As you have shown interest in this feature and submitted the 
>>>>>>>>>> design document, do you have any opinion on this? Do you see 
>>>>>>>>>> any merit in using D-Bus match in bmcweb to create event logs 
>>>>>>>>>> for life cycle events?  Please feel free to weigh in.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Rajes
>>>>>>>>>>
>>>>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I am going through the bmcweb code for implementing Redfish 
>>>>>>>>>>> EventService based on the design document 
>>>>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. 
>>>>>>>>>>> This design is hooked to the journal based Redfish Event 
>>>>>>>>>>> Logging. For life cycle events(ResourceAdded, 
>>>>>>>>>>> ResourceRemoved, ResourceUpdated),  using D-Bus match, 
>>>>>>>>>>> bmcweb can create an event log. This requires a JSON 
>>>>>>>>>>> dictionary, comprising an array of Redfish Resource Name and 
>>>>>>>>>>> the D-Bus path. This approach works only in case of one to 
>>>>>>>>>>> one mapping of Redfish Resource Name and the D-Bus path. For 
>>>>>>>>>>> propertiesChanged events, if the Redfish Resource property 
>>>>>>>>>>> is not on the same D-Bus path or the Redfish Resource 
>>>>>>>>>>> property name is different from the D-Bus property name, 
>>>>>>>>>>> then an additional JSON dictionary to maintain this 
>>>>>>>>>>> information is required. With D-Bus match alone in the 
>>>>>>>>>>> bmcweb, Redfish EventService can't be fully supported. For 
>>>>>>>>>>> the Message Registers and the Resource Types that are 
>>>>>>>>>>> supported, the relevant OpenBMC application must create an 
>>>>>>>>>>> event log in the journal using either the 
>>>>>>>>>>> phosphor::logging::entry or sd_journal_send() command.
>>>>>>>>>>>
>>>>>>>>>>> After realizing that with D-Bus match in the bmcweb alone 
>>>>>>>>>>> can't help to fully implement EventService, I prefer to 
>>>>>>>>>>> avoid using D-Bus match in bmcweb. Instead, I prefer to 
>>>>>>>>>>> modify the OpenBMC application that generated the event to 
>>>>>>>>>>> create an event log in the journal. Do you see any advantage 
>>>>>>>>>>> of using combination of D-Bus match in the bmcweb wherever 
>>>>>>>>>>> it is possible and changes to OpenBMC application in other 
>>>>>>>>>>> cases to create an event log ?
>>>>>>>>>>>
>>>>>>>>>>> Your views are highly appreciated.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Rajes
>>>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Ratan
>>>>>>>>
>>>>>>>>
>> Ratan

[-- Attachment #2: Type: text/html, Size: 24283 bytes --]

Re: Redfish EventService Implementation

[Brad Bishop]

On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote:
> Hi,
> 
> I am going through the bmcweb code for implementing Redfish
> EventService based on the design document 
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design
> is hooked to the journal based Redfish Event Logging.

Would anyone else be willing to opine on whether or not they think
journal based event schemes are what we want going forward for OpenBMC?

My feeling is that they are not - as an alternative IPC mechanism don't
we end up re-implementing things that DBus already does?  Doesn't it
require us to raise the same event twice everywhere (Once with DBus, and
once in the journal)?  What does journal based eventing do that DBus
signals don't do?

Please poke holes.

thx - brad

Re: Redfish EventService Implementation

[James Feist]

On 6/8/2020 2:08 PM, Brad Bishop wrote:
> On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote:
>> Hi,
>>
>> I am going through the bmcweb code for implementing Redfish
>> EventService based on the design document
>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design
>> is hooked to the journal based Redfish Event Logging.
> 
> Would anyone else be willing to opine on whether or not they think
> journal based event schemes are what we want going forward for OpenBMC?
> 
> My feeling is that they are not - as an alternative IPC mechanism don't
> we end up re-implementing things that DBus already does?  Doesn't it
> require us to raise the same event twice everywhere (Once with DBus, and
> once in the journal)?  What does journal based eventing do that DBus
> signals don't do?

We don't host log events on DBus at all, so there is no duplicate. The 
journal gives built in persistence and rotating of logs for large number 
of events. I know when this came up the last time one of the big issues 
was supporting thousands of logs wouldn't work well on DBus.


> 
> Please poke holes.
> 
> thx - brad
> 

Re: Redfish EventService Implementation

[Ratan Gupta]

Hi James/Apparao/Brad,

I am inclining towards having a separate application for Redfish 
Logs(like: phosphor-sel-logger),
This application does the following.

1) Have the mapping info from Redfish resources to Dbus Resources
      * This is needed as webserver have to provide the event filtering 
through Resource Type
      * eg : Redfish Client may ask as the client is interested in 
"Account" Resource type
             i.e all the user account related updates should be given to 
redfish client.
             which suggest that there should be mapping from Redfish 
Resource to Dbus Resource

2) Have the reverse mapping from Dbus Resources to Redfish Resources
      * This is needed to send the Redfish event if there is any changes 
in the
        corresponding D-bus resources. eg BMC state change/network 
change etc.

3) This application monitors the D-bus event and Log the event in the 
journal like below
     eg:
         sd_journal_send("MESSAGE=%s", "Account Modified",
             "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
"REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
             "REDFISH_RESOURCE_TYPE=ComputerSystem"
             "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
             "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);

4) rsyslogd will put all these logs from journal into "/var/log/redfish" 
file.

5) bmcweb/webserver would inotify this file location and on notification 
it will send redfish event

6) Event filtering would be done at the bmcweb/webserver side.


We already have the infrastructure for seq no 4 and seq no 5 and we 
wanted to leverage this infrastructure.

Please let me know if there is any concern with this approach.

Ratan

On 6/9/20 6:28 AM, James Feist wrote:
> On 6/8/2020 2:08 PM, Brad Bishop wrote:
>> On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote:
>>> Hi,
>>>
>>> I am going through the bmcweb code for implementing Redfish
>>> EventService based on the design document
>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design
>>> is hooked to the journal based Redfish Event Logging.
>>
>> Would anyone else be willing to opine on whether or not they think
>> journal based event schemes are what we want going forward for OpenBMC?
>>
>> My feeling is that they are not - as an alternative IPC mechanism don't
>> we end up re-implementing things that DBus already does? Doesn't it
>> require us to raise the same event twice everywhere (Once with DBus, and
>> once in the journal)?  What does journal based eventing do that DBus
>> signals don't do?
>
> We don't host log events on DBus at all, so there is no duplicate. The 
> journal gives built in persistence and rotating of logs for large 
> number of events. I know when this came up the last time one of the 
> big issues was supporting thousands of logs wouldn't work well on DBus.
>
>
>>
>> Please poke holes.
>>
>> thx - brad
>>

Re: Redfish EventService Implementation

[James Feist]

On 6/15/2020 5:50 AM, Ratan Gupta wrote:
> Hi James/Apparao/Brad,
> 
> I am inclining towards having a separate application for Redfish 
> Logs(like: phosphor-sel-logger),
> This application does the following.
> 
> 1) Have the mapping info from Redfish resources to Dbus Resources
>       * This is needed as webserver have to provide the event filtering 
> through Resource Type
>       * eg : Redfish Client may ask as the client is interested in 
> "Account" Resource type
>              i.e all the user account related updates should be given to 
> redfish client.
>              which suggest that there should be mapping from Redfish 
> Resource to Dbus Resource
> 
> 2) Have the reverse mapping from Dbus Resources to Redfish Resources
>       * This is needed to send the Redfish event if there is any changes 
> in the
>         corresponding D-bus resources. eg BMC state change/network 
> change etc.
> 
> 3) This application monitors the D-bus event and Log the event in the 
> journal like below

Why do we need a listener on d-bus if the application creating the event 
already knows all the information? Why can't it log straight to the journal?

>      eg:
>          sd_journal_send("MESSAGE=%s", "Account Modified",
>              "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
>              "REDFISH_RESOURCE_TYPE=ComputerSystem"
>              "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
>              "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);

If we're going to go down the path of re-implementing logging, I think 
the goal should be to stop logging things in the Journal that are 
Redfish specific, and instead log them in some generic format that 
phosphor logging controls the map. Right now we are bifurcated because 
the dbus-event-logs, SEL, PEL, and Redfish are all using different 
methods of logging, that play to their own set of rules. Most repos use 
phosphor-logging, so instead of creating yet another daemon, if we added 
support to create a 'System Level' or 'External User' log that has 
predefined dictionaries of required and optional keys, something like 
phosphor-dbus-interfaces, we might be able to drop some of these 
transport specific logs, and have the transports based on the same 
database (the journal). Then each transport could filter these based on 
journal entry type, and transform them into the correct log for that 
transport. I think adding more Redfish specifics into the logs hinders 
those who do not want Redfish in their systems.

> 
> 4) rsyslogd will put all these logs from journal into "/var/log/redfish" 
> file.
> 
> 5) bmcweb/webserver would inotify this file location and on notification 
> it will send redfish event
> 
> 6) Event filtering would be done at the bmcweb/webserver side.
> 

I think this will probably be needed if we want one source of truth for 
logs.

> 
> We already have the infrastructure for seq no 4 and seq no 5 and we 
> wanted to leverage this infrastructure.
> 
> Please let me know if there is any concern with this approach.
> 
> Ratan
> 
> On 6/9/20 6:28 AM, James Feist wrote:
>> On 6/8/2020 2:08 PM, Brad Bishop wrote:
>>> On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote:
>>>> Hi,
>>>>
>>>> I am going through the bmcweb code for implementing Redfish
>>>> EventService based on the design document
>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design
>>>> is hooked to the journal based Redfish Event Logging.
>>>
>>> Would anyone else be willing to opine on whether or not they think
>>> journal based event schemes are what we want going forward for OpenBMC?
>>>
>>> My feeling is that they are not - as an alternative IPC mechanism don't
>>> we end up re-implementing things that DBus already does? Doesn't it
>>> require us to raise the same event twice everywhere (Once with DBus, and
>>> once in the journal)?  What does journal based eventing do that DBus
>>> signals don't do?
>>
>> We don't host log events on DBus at all, so there is no duplicate. The 
>> journal gives built in persistence and rotating of logs for large 
>> number of events. I know when this came up the last time one of the 
>> big issues was supporting thousands of logs wouldn't work well on DBus.
>>
>>
>>>
>>> Please poke holes.
>>>
>>> thx - brad
>>>

Re: Redfish EventService Implementation

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 2365 bytes --]

On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote:
> On 6/15/2020 5:50 AM, Ratan Gupta wrote:
> >      eg:
> >          sd_journal_send("MESSAGE=%s", "Account Modified",
> >              "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
> > "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
> >              "REDFISH_RESOURCE_TYPE=ComputerSystem"
> >              "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
> >              "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
> 
> If we're going to go down the path of re-implementing logging, I think 
> the goal should be to stop logging things in the Journal that are 
> Redfish specific, and instead log them in some generic format that 
> phosphor logging controls the map. Right now we are bifurcated because 
> the dbus-event-logs, SEL, PEL, and Redfish are all using different 
> methods of logging, that play to their own set of rules. 

Absolutely agree with you here.  There is zero reason that applications
should start logging specially formed messages with REDFISH_* meta-data.
We shouldn't have any applications explicitly know about Redfish except
the Redfish providers themselves.  This is no different from IPMI, PLDM,
or any other external interface.

The kind of information presented here as being interesting to expose
via Redfish is equally as interesting to me to be able to add to one of
our 'FFDC dumps', which could be used for security / forensic work.

> Most repos use 
> phosphor-logging, so instead of creating yet another daemon, if we added 
> support to create a 'System Level' or 'External User' log that has 
> predefined dictionaries of required and optional keys, something like 
> phosphor-dbus-interfaces, we might be able to drop some of these 
> transport specific logs, and have the transports based on the same 
> database (the journal). Then each transport could filter these based on 
> journal entry type, and transform them into the correct log for that 
> transport. I think adding more Redfish specifics into the logs hinders 
> those who do not want Redfish in their systems.

Can't we do this already today by defining a simple errors/metadata file
in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
This will create a record on dbus in phosphor-logging.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Redfish EventService Implementation

[James Feist]

On 6/16/2020 8:24 AM, Patrick Williams wrote:
> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote:
>> On 6/15/2020 5:50 AM, Ratan Gupta wrote:
>>>       eg:
>>>           sd_journal_send("MESSAGE=%s", "Account Modified",
>>>               "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
>>>               "REDFISH_RESOURCE_TYPE=ComputerSystem"
>>>               "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
>>>               "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
>>
>> If we're going to go down the path of re-implementing logging, I think
>> the goal should be to stop logging things in the Journal that are
>> Redfish specific, and instead log them in some generic format that
>> phosphor logging controls the map. Right now we are bifurcated because
>> the dbus-event-logs, SEL, PEL, and Redfish are all using different
>> methods of logging, that play to their own set of rules.
> 
> Absolutely agree with you here.  There is zero reason that applications
> should start logging specially formed messages with REDFISH_* meta-data.
> We shouldn't have any applications explicitly know about Redfish except
> the Redfish providers themselves.  This is no different from IPMI, PLDM,
> or any other external interface.
> 
> The kind of information presented here as being interesting to expose
> via Redfish is equally as interesting to me to be able to add to one of
> our 'FFDC dumps', which could be used for security / forensic work.
> 
>> Most repos use
>> phosphor-logging, so instead of creating yet another daemon, if we added
>> support to create a 'System Level' or 'External User' log that has
>> predefined dictionaries of required and optional keys, something like
>> phosphor-dbus-interfaces, we might be able to drop some of these
>> transport specific logs, and have the transports based on the same
>> database (the journal). Then each transport could filter these based on
>> journal entry type, and transform them into the correct log for that
>> transport. I think adding more Redfish specifics into the logs hinders
>> those who do not want Redfish in their systems.
> 
> Can't we do this already today by defining a simple errors/metadata file
> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
> This will create a record on dbus in phosphor-logging.
> 
I think the original concern was with supporting on the order of 10,000 
log entries, having this on d-bus seemed impractical. Also the free log 
rotation the journal provides is useful. Now modifying the 
logging::report<...> to conditionally log to the journal seems realistic.

Re: Redfish EventService Implementation

[Ratan Gupta]

Hi James,Pattrick.

Thanks for giving the feedback

On 6/16/20 9:41 PM, James Feist wrote:
> On 6/16/2020 8:24 AM, Patrick Williams wrote:
>> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote:
>>> On 6/15/2020 5:50 AM, Ratan Gupta wrote:
>>>>       eg:
>>>>           sd_journal_send("MESSAGE=%s", "Account Modified",
>>>>               "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
>>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
>>>>               "REDFISH_RESOURCE_TYPE=ComputerSystem"
>>>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
>>>>               "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
>>>
>>> If we're going to go down the path of re-implementing logging, I think
>>> the goal should be to stop logging things in the Journal that are
>>> Redfish specific, and instead log them in some generic format that
>>> phosphor logging controls the map. Right now we are bifurcated because
>>> the dbus-event-logs, SEL, PEL, and Redfish are all using different
>>> methods of logging, that play to their own set of rules.
>>
>> Absolutely agree with you here.  There is zero reason that applications
>> should start logging specially formed messages with REDFISH_* meta-data.
>> We shouldn't have any applications explicitly know about Redfish except
>> the Redfish providers themselves.  This is no different from IPMI, PLDM,
>> or any other external interface.
>>
>> The kind of information presented here as being interesting to expose
>> via Redfish is equally as interesting to me to be able to add to one of
>> our 'FFDC dumps', which could be used for security / forensic work.
>>
>>> Most repos use
>>> phosphor-logging, so instead of creating yet another daemon, if we 
>>> added
>>> support to create a 'System Level' or 'External User' log that has
>>> predefined dictionaries of required and optional keys, something like
>>> phosphor-dbus-interfaces, we might be able to drop some of these
>>> transport specific logs, and have the transports based on the same
>>> database (the journal). Then each transport could filter these based on
>>> journal entry type, and transform them into the correct log for that
>>> transport. I think adding more Redfish specifics into the logs hinders
>>> those who do not want Redfish in their systems.
>>
>> Can't we do this already today by defining a simple errors/metadata file
>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>> This will create a record on dbus in phosphor-logging.
>>
> I think the original concern was with supporting on the order of 
> 10,000 log entries, having this on d-bus seemed impractical. Also the 
> free log rotation the journal provides is useful. Now modifying the 
> logging::report<...> to conditionally log to the journal seems realistic.
My intention was not to re-implement the logging, my intention was to 
extend/use the existing design which we already have it below.

https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md

I was trying not to bring the Redfish specific stuff in each individual 
repo, instead each transport can listen for
Dbus events and write to the journal which goes to their app specific file.

Your point is valid that we would be polluting the journal if each and 
every transport start writing as per their needs.

***** As per the Redfish one of the requirement is we need the log for 
most of the Dbus Property update/interface added as they
are mapped to some Redfish Resource and the bmcweb has to send the 
Resource updated/modified signal to the
Redfish client. ******

As we are in agreement that we want to use the journal for persistence 
and log rotate feature.

We have two options:
     1) Each transport interface listens for the Dbus signals and write 
it to their app specific file.
     2) Each openbmc repo must use log::report for each D-bus property 
update/ interface added.
        To make this happen, we need the following
         * common message format which can be used by the all other 
application (SEL,Redfish) etc.
         * Transport application will read this common message from the 
journal and use it as per their needs.

Option no 2:   It is ideal solution but that requires good amount of 
work as
              either each and every D-bus repo  should write to the 
journal for any property update/interface added.
                      or
              Should SDbusplus do this as each and every application 
uses the sdbusplus framework to manage the dbus objects

Please let me know your thoughts.

Ratan

Re: Redfish EventService Implementation

[James Feist]

> My intention was not to re-implement the logging, my intention was to 
> extend/use the existing design which we already have it below.
> 
> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md 
> 
> 
> I was trying not to bring the Redfish specific stuff in each individual 
> repo, instead each transport can listen for
> Dbus events and write to the journal which goes to their app specific file.
> 
> Your point is valid that we would be polluting the journal if each and 
> every transport start writing as per their needs.
> 
> ***** As per the Redfish one of the requirement is we need the log for 
> most of the Dbus Property update/interface added as they
> are mapped to some Redfish Resource and the bmcweb has to send the 
> Resource updated/modified signal to the
> Redfish client. ******
> 
> As we are in agreement that we want to use the journal for persistence 
> and log rotate feature.
> 
> We have two options:
>      1) Each transport interface listens for the Dbus signals and write 
> it to their app specific file.
>      2) Each openbmc repo must use log::report for each D-bus property 
> update/ interface added.

I'm assuming you meant each event added right? I don't see any reason 
that the logs have anything to do with d-bus, except maybe if it seems 
worth it for phosphor-logging to catch all the reports and log them to 
the journal. Although it would probably be more efficient (and easier 
for phased adoption) to just make the log::report call write to the 
journal as well.

>         To make this happen, we need the following
>          * common message format which can be used by the all other 
> application (SEL,Redfish) etc.

Agree.

>          * Transport application will read this common message from the 
> journal and use it as per their needs.
> 
> Option no 2:   It is ideal solution but that requires good amount of 
> work as
>               either each and every D-bus repo  should write to the 
> journal for any property update/interface added.

Why every property/interface added? This would make the logs much to 
large to be useful. And many times the information needed is specific to 
the log type. For instance a event for inventory discovered might need 
the type of inventory, the serial number, the model, etc. To get all 
that information would be multiple interfaces added events combined 
together. It would make more sense in this aspect for the inventory 
daemon to take the parts out of say the Asset, and Chassis interfaces 
along with the path identifier, and create an entry in the log that is 
use-able. It would be much more difficult and error-prone for the 
Redfish or SEL daemon to try to piece these multiple events together to 
form something useful.

>                       or
>               Should SDbusplus do this as each and every application 
> uses the sdbusplus framework to manage the dbus objects
> 
> Please let me know your thoughts.
> 
> Ratan
> 

Re: Redfish EventService Implementation

[Bills, Jason M]

On 6/17/2020 5:08 AM, Ratan Gupta wrote:
> Hi James,Pattrick.
> 
> Thanks for giving the feedback
> 
> On 6/16/20 9:41 PM, James Feist wrote:
>> On 6/16/2020 8:24 AM, Patrick Williams wrote:
>>> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote:
>>>> On 6/15/2020 5:50 AM, Ratan Gupta wrote:
>>>>>       eg:
>>>>>           sd_journal_send("MESSAGE=%s", "Account Modified",
>>>>>               "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
>>>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
>>>>>               "REDFISH_RESOURCE_TYPE=ComputerSystem"
>>>>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
>>>>>               "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
>>>>
>>>> If we're going to go down the path of re-implementing logging, I think
>>>> the goal should be to stop logging things in the Journal that are
>>>> Redfish specific, and instead log them in some generic format that
>>>> phosphor logging controls the map. Right now we are bifurcated because
>>>> the dbus-event-logs, SEL, PEL, and Redfish are all using different
>>>> methods of logging, that play to their own set of rules.
>>>
>>> Absolutely agree with you here.  There is zero reason that applications
>>> should start logging specially formed messages with REDFISH_* meta-data.
>>> We shouldn't have any applications explicitly know about Redfish except
>>> the Redfish providers themselves.  This is no different from IPMI, PLDM,
>>> or any other external interface.
>>>
>>> The kind of information presented here as being interesting to expose
>>> via Redfish is equally as interesting to me to be able to add to one of
>>> our 'FFDC dumps', which could be used for security / forensic work.
>>>
>>>> Most repos use
>>>> phosphor-logging, so instead of creating yet another daemon, if we 
>>>> added
>>>> support to create a 'System Level' or 'External User' log that has
>>>> predefined dictionaries of required and optional keys, something like
>>>> phosphor-dbus-interfaces, we might be able to drop some of these
>>>> transport specific logs, and have the transports based on the same
>>>> database (the journal). Then each transport could filter these based on
>>>> journal entry type, and transform them into the correct log for that
>>>> transport. I think adding more Redfish specifics into the logs hinders
>>>> those who do not want Redfish in their systems.
>>>
>>> Can't we do this already today by defining a simple errors/metadata file
>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>>> This will create a record on dbus in phosphor-logging.
>>>
>> I think the original concern was with supporting on the order of 
>> 10,000 log entries, having this on d-bus seemed impractical. Also the 
>> free log rotation the journal provides is useful. Now modifying the 
>> logging::report<...> to conditionally log to the journal seems realistic.
> My intention was not to re-implement the logging, my intention was to 
> extend/use the existing design which we already have it below.
> 
> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md 
> 
> 
> I was trying not to bring the Redfish specific stuff in each individual 
> repo, instead each transport can listen for
> Dbus events and write to the journal which goes to their app specific file.
> 
> Your point is valid that we would be polluting the journal if each and 
> every transport start writing as per their needs.
> 
> ***** As per the Redfish one of the requirement is we need the log for 
> most of the Dbus Property update/interface added as they
> are mapped to some Redfish Resource and the bmcweb has to send the 
> Resource updated/modified signal to the
> Redfish client. ******
Why do we want to log on D-Bus property updates?  This seems like it 
will be too noisy for the EventLog.

> 
> As we are in agreement that we want to use the journal for persistence 
> and log rotate feature.
> 
> We have two options:
>      1) Each transport interface listens for the Dbus signals and write 
> it to their app specific file.
>      2) Each openbmc repo must use log::report for each D-bus property 
> update/ interface added.
>         To make this happen, we need the following
>          * common message format which can be used by the all other 
> application (SEL,Redfish) etc.
>          * Transport application will read this common message from the 
> journal and use it as per their needs.
> 
> Option no 2:   It is ideal solution but that requires good amount of 
> work as
>               either each and every D-bus repo  should write to the 
> journal for any property update/interface added.
>                       or
>               Should SDbusplus do this as each and every application 
> uses the sdbusplus framework to manage the dbus objects
> 
> Please let me know your thoughts.
> 
> Ratan
> 

Re: Redfish EventService Implementation

[Andrew Geissler]

> On Jun 17, 2020, at 12:19 PM, Bills, Jason M <jason.m.bills@linux.intel.com> wrote:
> 
> 
> 
> On 6/17/2020 5:08 AM, Ratan Gupta wrote:
>> Hi James,Pattrick.
>> Thanks for giving the feedback
>> On 6/16/20 9:41 PM, James Feist wrote:
>>> On 6/16/2020 8:24 AM, Patrick Williams wrote:
>>>> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote:
>>>>> On 6/15/2020 5:50 AM, Ratan Gupta wrote:
>>>>>>       eg:
>>>>>>           sd_journal_send("MESSAGE=%s", "Account Modified",
>>>>>>               "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
>>>>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
>>>>>>               "REDFISH_RESOURCE_TYPE=ComputerSystem"
>>>>>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
>>>>>>               "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
>>>>> 
>>>>> If we're going to go down the path of re-implementing logging, I think
>>>>> the goal should be to stop logging things in the Journal that are
>>>>> Redfish specific, and instead log them in some generic format that
>>>>> phosphor logging controls the map. Right now we are bifurcated because
>>>>> the dbus-event-logs, SEL, PEL, and Redfish are all using different
>>>>> methods of logging, that play to their own set of rules.
>>>> 
>>>> Absolutely agree with you here.  There is zero reason that applications
>>>> should start logging specially formed messages with REDFISH_* meta-data.
>>>> We shouldn't have any applications explicitly know about Redfish except
>>>> the Redfish providers themselves.  This is no different from IPMI, PLDM,
>>>> or any other external interface.
>>>> 
>>>> The kind of information presented here as being interesting to expose
>>>> via Redfish is equally as interesting to me to be able to add to one of
>>>> our 'FFDC dumps', which could be used for security / forensic work.
>>>> 
>>>>> Most repos use
>>>>> phosphor-logging, so instead of creating yet another daemon, if we added
>>>>> support to create a 'System Level' or 'External User' log that has
>>>>> predefined dictionaries of required and optional keys, something like
>>>>> phosphor-dbus-interfaces, we might be able to drop some of these
>>>>> transport specific logs, and have the transports based on the same
>>>>> database (the journal). Then each transport could filter these based on
>>>>> journal entry type, and transform them into the correct log for that
>>>>> transport. I think adding more Redfish specifics into the logs hinders
>>>>> those who do not want Redfish in their systems.
>>>> 
>>>> Can't we do this already today by defining a simple errors/metadata file
>>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>>>> This will create a record on dbus in phosphor-logging.
>>>> 
>>> I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic.
>> My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below.
>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for
>> Dbus events and write to the journal which goes to their app specific file.
>> Your point is valid that we would be polluting the journal if each and every transport start writing as per their needs.
>> ***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they
>> are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the
>> Redfish client. ******
> Why do we want to log on D-Bus property updates?  This seems like it will be too noisy for the EventLog.

I don’t think it would be all, just the one’s the different transport layers need.

So to get my head straight on this, there is a commit in state-manager to log
the redfish events to the journal:
https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-state-manager/+/33494

With what Ratan proposed above, we have two options?
1) A separate application that listens for the chassis properties to change and logs those
    events to the journal in the format the transport layer wants
2) Code within phosphor-state-manager that logs a generic event to the
    journal which then can be post processed by the different transport 
    layer code.

> 
>> As we are in agreement that we want to use the journal for persistence and log rotate feature.
>> We have two options:
>>     1) Each transport interface listens for the Dbus signals and write it to their app specific file.
>>     2) Each openbmc repo must use log::report for each D-bus property update/ interface added.
>>        To make this happen, we need the following
>>         * common message format which can be used by the all other application (SEL,Redfish) etc.
>>         * Transport application will read this common message from the journal and use it as per their needs.
>> Option no 2:   It is ideal solution but that requires good amount of work as
>>              either each and every D-bus repo  should write to the journal for any property update/interface added.
>>                      or
>>              Should SDbusplus do this as each and every application uses the sdbusplus framework to manage the dbus objects
>> Please let me know your thoughts.
>> Ratan

Re: Redfish EventService Implementation

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 3658 bytes --]

On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote:
> Hi James,Pattrick.
> 
> >> Can't we do this already today by defining a simple errors/metadata file
> >> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
> >> This will create a record on dbus in phosphor-logging.
> >>
> > I think the original concern was with supporting on the order of 
> > 10,000 log entries, having this on d-bus seemed impractical. Also the 
> > free log rotation the journal provides is useful. Now modifying the 
> > logging::report<...> to conditionally log to the journal seems realistic.
> My intention was not to re-implement the logging, my intention was to 
> extend/use the existing design which we already have it below.
> 
> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
> 
> I was trying not to bring the Redfish specific stuff in each individual 
> repo, instead each transport can listen for
> Dbus events and write to the journal which goes to their app specific file.

Good.  This wasn't clear from the earlier email.  Thanks.


> As we are in agreement that we want to use the journal for persistence 
> and log rotate feature.

I'm not convinced there is agreement on this.  There has been
disagreement about even using the journal for phosphor-logging use since
the beginning and I suspect there would be less agreement on another
application using it as its own IPC mechanism.

Just because a hammer can be used to insert a nail into a board doesn't
mean you use it insert any pointy thing into a flat thing.  [ Just
because the journal provides log rotation and persistance doesn't mean
you should use it for every feature needing log rotation and
persistance. ]


> ***** As per the Redfish one of the requirement is we need the log for 
> most of the Dbus Property update/interface added as they
> are mapped to some Redfish Resource and the bmcweb has to send the 
> Resource updated/modified signal to the
> Redfish client. ******

I don't know Redfish well, so bear with me if there is something obvious
I'm missing.  But, the first part of this "requirement" doesn't seem to
follow from the second part of the "requirement" to me.

Sending a signal of a property changing to the Redfish clients is
straight-forwawrd; Redfish should subscribe to all the appropriate
dbus-events.  I don't understand how this implies any sort of logging.
Where does the logging part of this requirement come from?

> We have two options:
>      1) Each transport interface listens for the Dbus signals and write 
> it to their app specific file.
>      2) Each openbmc repo must use log::report for each D-bus property 
> update/ interface added.

#2 is absolutely unworkable on the surface to me.  log::report is to
create a error entry (xyz.openbmc_project.Logging.Entry), which creates
a dbus-object, which would cause log::report to be called, which creates
a dbus-object, which ...

Even if what you meant was something like logging::log<info>, this seems
pretty heavy.  I'm not sure this is something that can be inserted into
sdbusplus, especially for the ASIO-based object servers, because in many
cases applications register their own callback.  For the sdbus++
generated server bindings we could squeeze it in.  But, what you're
proposing here is essentially a "journal-as-dbus-monitor".  We'd
certainly need to make some measurements on how many kB/s worth of
journal entries this would create because I suspect we could end up
burning out a NAND flash with as many writes as that would induce.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Redfish EventService Implementation

[Ratan Gupta]

[-- Attachment #1: Type: text/plain, Size: 5106 bytes --]


On 6/18/20 2:15 AM, Patrick Williams wrote:
> On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote:
>> Hi James,Pattrick.
>>
>>>> Can't we do this already today by defining a simple errors/metadata file
>>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>>>> This will create a record on dbus in phosphor-logging.
>>>>
>>> I think the original concern was with supporting on the order of
>>> 10,000 log entries, having this on d-bus seemed impractical. Also the
>>> free log rotation the journal provides is useful. Now modifying the
>>> logging::report<...> to conditionally log to the journal seems realistic.
>> My intention was not to re-implement the logging, my intention was to
>> extend/use the existing design which we already have it below.
>>
>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
>>
>> I was trying not to bring the Redfish specific stuff in each individual
>> repo, instead each transport can listen for
>> Dbus events and write to the journal which goes to their app specific file.
> Good.  This wasn't clear from the earlier email.  Thanks.
>
>
>> As we are in agreement that we want to use the journal for persistence
>> and log rotate feature.
> I'm not convinced there is agreement on this.  There has been
> disagreement about even using the journal for phosphor-logging use since
> the beginning and I suspect there would be less agreement on another
> application using it as its own IPC mechanism.
>
> Just because a hammer can be used to insert a nail into a board doesn't
> mean you use it insert any pointy thing into a flat thing.  [ Just
> because the journal provides log rotation and persistance doesn't mean
> you should use it for every feature needing log rotation and
> persistance. ]
>
>
>> ***** As per the Redfish one of the requirement is we need the log for
>> most of the Dbus Property update/interface added as they
>> are mapped to some Redfish Resource and the bmcweb has to send the
>> Resource updated/modified signal to the
>> Redfish client. ******

Jaosn: You asked the following query in other thread /*"Why do we want 
to log on D-Bus property updates?  This seems like it will be too noisy 
for the EventLog*"/

Eg: Client is interested for an event when ever there is any user 
add/delete or network configuration change or there is a log entry 
resource gets created,To handle this request the flow would be

Redfish Client subscribe for "ResourceType" eg: 
"EthernetService,AccountService,LogService"  with subordinate resources 
property as truewhich means the Client is looking for updates on the 
subscribed resources and the subordinates resource, These redfish 
resources(EthernetInterface, IP address, ManagerAccount, AccountService) 
would be mapped to some D-bus Resources, hence some application/bmcweb  
would monitor the Dbus signals on the interested Dbus objects and send 
the Redfish event to the subscribed client.

Apparo: Please correct me if I am missing something.

> I don't know Redfish well, so bear with me if there is something obvious
> I'm missing.  But, the first part of this "requirement" doesn't seem to
> follow from the second part of the "requirement" to me.
>
> Sending a signal of a property changing to the Redfish clients is
> straight-forwawrd; Redfish should subscribe to all the appropriate
> dbus-events.  I don't understand how this implies any sort of logging.
> Where does the logging part of this requirement come from?
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474

While I am reading the redfish 
spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf) 
, it is not clear that the events need to be persisted.I will ask in the 
DMTF for the persistence of the events.

>> We have two options:
>>       1) Each transport interface listens for the Dbus signals and write
>> it to their app specific file.
>>       2) Each openbmc repo must use log::report for each D-bus property
>> update/ interface added.
> #2 is absolutely unworkable on the surface to me.  log::report is to
> create a error entry (xyz.openbmc_project.Logging.Entry), which creates
> a dbus-object, which would cause log::report to be called, which creates
> a dbus-object, which ...
>
> Even if what you meant was something like logging::log<info>, this seems
> pretty heavy.  I'm not sure this is something that can be inserted into
> sdbusplus, especially for the ASIO-based object servers, because in many
> cases applications register their own callback.  For the sdbus++
> generated server bindings we could squeeze it in.  But, what you're
> proposing here is essentially a "journal-as-dbus-monitor".  We'd
> certainly need to make some measurements on how many kB/s worth of
> journal entries this would create because I suspect we could end up
> burning out a NAND flash with as many writes as that would induce.
I would respond on the same once my query gets answered from DMTF.
If my query gets answered yes then we have to write on flash but let's
wait for it,

[-- Attachment #2: Type: text/html, Size: 7343 bytes --]

Re: Redfish EventService Implementation

[Bills, Jason M]

On 6/19/2020 6:26 AM, Ratan Gupta wrote:
> 
> On 6/18/20 2:15 AM, Patrick Williams wrote:
>> On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote:
>>> Hi James,Pattrick.
>>>
>>>>> Can't we do this already today by defining a simple errors/metadata file
>>>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>>>>> This will create a record on dbus in phosphor-logging.
>>>>>
>>>> I think the original concern was with supporting on the order of
>>>> 10,000 log entries, having this on d-bus seemed impractical. Also the
>>>> free log rotation the journal provides is useful. Now modifying the
>>>> logging::report<...> to conditionally log to the journal seems realistic.
>>> My intention was not to re-implement the logging, my intention was to
>>> extend/use the existing design which we already have it below.
>>>
>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
>>>
>>> I was trying not to bring the Redfish specific stuff in each individual
>>> repo, instead each transport can listen for
>>> Dbus events and write to the journal which goes to their app specific file.
>> Good.  This wasn't clear from the earlier email.  Thanks.
>>
>>
>>> As we are in agreement that we want to use the journal for persistence
>>> and log rotate feature.
>> I'm not convinced there is agreement on this.  There has been
>> disagreement about even using the journal for phosphor-logging use since
>> the beginning and I suspect there would be less agreement on another
>> application using it as its own IPC mechanism.
>>
>> Just because a hammer can be used to insert a nail into a board doesn't
>> mean you use it insert any pointy thing into a flat thing.  [ Just
>> because the journal provides log rotation and persistance doesn't mean
>> you should use it for every feature needing log rotation and
>> persistance. ]
>>
>>
>>> ***** As per the Redfish one of the requirement is we need the log for
>>> most of the Dbus Property update/interface added as they
>>> are mapped to some Redfish Resource and the bmcweb has to send the
>>> Resource updated/modified signal to the
>>> Redfish client. ******
> 
> Jaosn: You asked the following query in other thread /*"Why do we want 
> to log on D-Bus property updates?  This seems like it will be too noisy 
> for the EventLog*"/
> 
> Eg: Client is interested for an event when ever there is any user 
> add/delete or network configuration change or there is a log entry 
> resource gets created,To handle this request the flow would be
My understanding is for the first iteration we are only providing 
notifications for event messages.  So, if it isn't in the message 
registry, then you cannot register for that event?

It seems like the full resource change event notification will be a 
major feature and should probably get a dedicated design.

> 
> Redfish Client subscribe for "ResourceType" eg: 
> "EthernetService,AccountService,LogService"  with subordinate resources 
> property as truewhich means the Client is looking for updates on the 
> subscribed resources and the subordinates resource, These redfish 
> resources(EthernetInterface, IP address, ManagerAccount, AccountService) 
> would be mapped to some D-bus Resources, hence some application/bmcweb  
> would monitor the Dbus signals on the interested Dbus objects and send 
> the Redfish event to the subscribed client.
> 
> Apparo: Please correct me if I am missing something.
> 
>> I don't know Redfish well, so bear with me if there is something obvious
>> I'm missing.  But, the first part of this "requirement" doesn't seem to
>> follow from the second part of the "requirement" to me.
>>
>> Sending a signal of a property changing to the Redfish clients is
>> straight-forwawrd; Redfish should subscribe to all the appropriate
>> dbus-events.  I don't understand how this implies any sort of logging.
>> Where does the logging part of this requirement come from?
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474
> 
> While I am reading the redfish 
> spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf) 
> , it is not clear that the events need to be persisted.I will ask in the 
> DMTF for the persistence of the events.
> 
>>> We have two options:
>>>       1) Each transport interface listens for the Dbus signals and write
>>> it to their app specific file.
>>>       2) Each openbmc repo must use log::report for each D-bus property
>>> update/ interface added.
>> #2 is absolutely unworkable on the surface to me.  log::report is to
>> create a error entry (xyz.openbmc_project.Logging.Entry), which creates
>> a dbus-object, which would cause log::report to be called, which creates
>> a dbus-object, which ...
>>
>> Even if what you meant was something like logging::log<info>, this seems
>> pretty heavy.  I'm not sure this is something that can be inserted into
>> sdbusplus, especially for the ASIO-based object servers, because in many
>> cases applications register their own callback.  For the sdbus++
>> generated server bindings we could squeeze it in.  But, what you're
>> proposing here is essentially a "journal-as-dbus-monitor".  We'd
>> certainly need to make some measurements on how many kB/s worth of
>> journal entries this would create because I suspect we could end up
>> burning out a NAND flash with as many writes as that would induce.
> I would respond on the same once my query gets answered from DMTF.
> If my query gets answered yes then we have to write on flash but let's
> wait for it,

Re: Redfish EventService Implementation

[Ratan Gupta]

[-- Attachment #1: Type: text/plain, Size: 6517 bytes --]

On 6/19/20 6:56 PM, Ratan Gupta wrote:
>
>
> On 6/18/20 2:15 AM, Patrick Williams wrote:
>> On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote:
>>> Hi James,Pattrick.
>>>
>>>>> Can't we do this already today by defining a simple errors/metadata file
>>>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>>>>> This will create a record on dbus in phosphor-logging.
>>>>>
>>>> I think the original concern was with supporting on the order of
>>>> 10,000 log entries, having this on d-bus seemed impractical. Also the
>>>> free log rotation the journal provides is useful. Now modifying the
>>>> logging::report<...> to conditionally log to the journal seems realistic.
>>> My intention was not to re-implement the logging, my intention was to
>>> extend/use the existing design which we already have it below.
>>>
>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
>>>
>>> I was trying not to bring the Redfish specific stuff in each individual
>>> repo, instead each transport can listen for
>>> Dbus events and write to the journal which goes to their app specific file.
>> Good.  This wasn't clear from the earlier email.  Thanks.
>>
>>
>>> As we are in agreement that we want to use the journal for persistence
>>> and log rotate feature.
>> I'm not convinced there is agreement on this.  There has been
>> disagreement about even using the journal for phosphor-logging use since
>> the beginning and I suspect there would be less agreement on another
>> application using it as its own IPC mechanism.
>>
>> Just because a hammer can be used to insert a nail into a board doesn't
>> mean you use it insert any pointy thing into a flat thing.  [ Just
>> because the journal provides log rotation and persistance doesn't mean
>> you should use it for every feature needing log rotation and
>> persistance. ]
>>
>>
>>> ***** As per the Redfish one of the requirement is we need the log for
>>> most of the Dbus Property update/interface added as they
>>> are mapped to some Redfish Resource and the bmcweb has to send the
>>> Resource updated/modified signal to the
>>> Redfish client. ******
>
> Jaosn: You asked the following query in other thread /*"Why do we want 
> to log on D-Bus property updates?  This seems like it will be too 
> noisy for the EventLog*"/
>
> Eg: Client is interested for an event when ever there is any user 
> add/delete or network configuration change or there is a log entry 
> resource gets created,To handle this request the flow would be
>
> Redfish Client subscribe for "ResourceType" eg: 
> "EthernetService,AccountService,LogService"  with subordinate 
> resources property as truewhich means the Client is looking for 
> updates on the subscribed resources and the subordinates resource, 
> These redfish resources(EthernetInterface, IP address, ManagerAccount, 
> AccountService) would be mapped to some D-bus Resources, hence some 
> application/bmcweb  would monitor the Dbus signals on the interested 
> Dbus objects and send the Redfish event to the subscribed client.
>
> Apparo: Please correct me if I am missing something.
>
>> I don't know Redfish well, so bear with me if there is something obvious
>> I'm missing.  But, the first part of this "requirement" doesn't seem to
>> follow from the second part of the "requirement" to me.
>>
>> Sending a signal of a property changing to the Redfish clients is
>> straight-forwawrd; Redfish should subscribe to all the appropriate
>> dbus-events.  I don't understand how this implies any sort of logging.
>> Where does the logging part of this requirement come from?
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474
>
> While I am reading the redfish 
> spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf) 
> , it is not clear that the events need to be persisted.I will ask in 
> the DMTF for the persistence of the events.
I have got an update from the DMTF that persistence is not required to 
be implemented for the 
events(https://redfish.dmtf.org/schemas/v1/Event.v1_5_0.json),
which implies we don't need to log the events in Journal.

More info(https://github.com/DMTF/Redfish/issues/3646)

Now updating the proposal where redfish client subscribed for Resource 
Type based events
1) Client would subscribe for the Redfish Resource(eg:ManagerAccount) to 
receive events like Account add/delete/modify
      Hence need for mapping from (RedfishResource to Dbus Resource)

2) Have the mapping info from Redfish resources to DBUS Resources (Some 
JSon file may have this mapping)

2) Have the reverse mapping from DBUS Resources to Redfish Resources
      * This is needed to send the Redfish event if there is any changes 
in the corresponding D-bus resources.
        eg BMC state change/network change etc.

3) bmcweb would monitor the DBUS events

4) Get the Redfish Path from the Mapping(2) and send the Redfish event

5) Bmcweb would buffer N number of events that can be configurable by 
redfish client. Buffer would get cleaned up in case of bmcweb restart or 
BMC reset.

James, Apparao: Please let me know if there is any more concern with 
this approach.
>
>>> We have two options:
>>>       1) Each transport interface listens for the Dbus signals and write
>>> it to their app specific file.
>>>       2) Each openbmc repo must use log::report for each D-bus property
>>> update/ interface added.
>> #2 is absolutely unworkable on the surface to me.  log::report is to
>> create a error entry (xyz.openbmc_project.Logging.Entry), which creates
>> a dbus-object, which would cause log::report to be called, which creates
>> a dbus-object, which ...
>>
>> Even if what you meant was something like logging::log<info>, this seems
>> pretty heavy.  I'm not sure this is something that can be inserted into
>> sdbusplus, especially for the ASIO-based object servers, because in many
>> cases applications register their own callback.  For the sdbus++
>> generated server bindings we could squeeze it in.  But, what you're
>> proposing here is essentially a "journal-as-dbus-monitor".  We'd
>> certainly need to make some measurements on how many kB/s worth of
>> journal entries this would create because I suspect we could end up
>> burning out a NAND flash with as many writes as that would induce.
> I would respond on the same once my query gets answered from DMTF.
> If my query gets answered yes then we have to write on flash but let's
> wait for it,

[-- Attachment #2: Type: text/html, Size: 9634 bytes --]

Re: Redfish EventService Implementation

[James Feist]

> I have got an update from the DMTF that persistence is not required to 
> be implemented for the 
> events(https://redfish.dmtf.org/schemas/v1/Event.v1_5_0.json),
> which implies we don't need to log the events in Journal.
> 
> More info(https://github.com/DMTF/Redfish/issues/3646)
> 
> Now updating the proposal where redfish client subscribed for Resource 
> Type based events
> 1) Client would subscribe for the Redfish Resource(eg:ManagerAccount) to 
> receive events like Account add/delete/modify
>       Hence need for mapping from (RedfishResource to Dbus Resource)
> 
> 2) Have the mapping info from Redfish resources to DBUS Resources (Some 
> JSon file may have this mapping)

This assumes the mapping is static, which on many systems it isn't, 
right? I think this needs to be developed to see what it would be like.

> 
> 2) Have the reverse mapping from DBUS Resources to Redfish Resources
>       * This is needed to send the Redfish event if there is any changes 
> in the corresponding D-bus resources.
>         eg BMC state change/network change etc.
> 
> 3) bmcweb would monitor the DBUS events

Which events specifically? It seems like it would be monitoring lots of 
events in this proposal. Some examples could be useful.

> 
> 4) Get the Redfish Path from the Mapping(2) and send the Redfish event
> 
> 5) Bmcweb would buffer N number of events that can be configurable by 
> redfish client. Buffer would get cleaned up in case of bmcweb restart or 
> BMC reset.
> 

Can you please push a patch to the event service design doc with the 
change to the design you'd like? Some examples in the design would help 
with understanding the proposal. This thread is quite long and hard to 
follow for someone not involved, it'd be good to document the proposed 
changes.

https://github.com/openbmc/docs/blob/master/designs/redfish-eventservice.md

Re: Redfish EventService Implementation

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 591 bytes --]

On Wed, Jun 24, 2020 at 09:24:03AM -0700, James Feist wrote:
> > 2) Have the mapping info from Redfish resources to DBUS Resources (Some 
> > JSon file may have this mapping)
> 
> This assumes the mapping is static, which on many systems it isn't, 
> right? I think this needs to be developed to see what it would be like.
> 

Don't we, in effect, have the mapping from Redfish to dbus by nature of
the Redfish providers that create their content from dbus objects?  Is
there any way to reuse this information to create the corresponding
Redfish events?

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Redfish EventService Implementation

[Brad Bishop]

On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote:
> > 3) bmcweb would monitor the DBUS events
> 
> Which events specifically? It seems like it would be monitoring lots
> of events in this proposal. 

Just curious what is the concern around monitoring lots of events?

Re: Redfish EventService Implementation

[James Feist]

On 6/25/2020 6:45 AM, Brad Bishop wrote:
> On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote:
>>> 3) bmcweb would monitor the DBUS events
>>
>> Which events specifically? It seems like it would be monitoring lots
>> of events in this proposal.
> 
> Just curious what is the concern around monitoring lots of events?
> 

Performance as always, depending on the number of matches needed. 
Although a POC could prove this not an issue.

Re: Redfish EventService Implementation

[Brad Bishop]

On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote:
> This thread is quite long and hard to follow for someone not involved,
> it'd be good to document the proposed changes.

You are right of course that this thread has become hard to follow.  I
think careful selection of reply context really would have helped.

FWIW I have asked the team here at IBM to hash out designs to some level
of consensus here on the list before making design submissions to gerrit
because IMO reviews of half baked designs in gerrit are even harder to
follow than email threads like this one 🙂

Re: Redfish EventService Implementation

[James Feist]

On 6/25/2020 8:49 AM, Brad Bishop wrote:
> On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote:
>> This thread is quite long and hard to follow for someone not involved,
>> it'd be good to document the proposed changes.
> 
> You are right of course that this thread has become hard to follow.  I
> think careful selection of reply context really would have helped.
> 
> FWIW I have asked the team here at IBM to hash out designs to some level
> of consensus here on the list before making design submissions to gerrit
> because IMO reviews of half baked designs in gerrit are even harder to
> follow than email threads like this one 🙂
> 

That's good. Was just hoping that the 'final design' wasn't this email, 
as it would be difficult to see we matched a design.

Re: Redfish EventService Implementation

[Brad Bishop]

On Tue, 2020-06-23 at 12:57 +0530, Ratan Gupta wrote:
> 1) Client would subscribe for the Redfish Resource(eg:ManagerAccount)
> to receive events like Account add/delete/modify.  Hence need for
> mapping from (RedfishResource to Dbus Resource)
> 
> 2) Have the mapping info from Redfish resources to DBUS Resources
> (Some JSon file may have this mapping)

It probably doesn't make sense to have any json files.  The application
logic itself does the mapping.

> 3) Have the reverse mapping from DBUS Resources to Redfish Resources
> This is needed to send the Redfish event if there is any changes in
> the corresponding D-bus resources. eg BMC state change/network change
> etc.
> 
> 4) bmcweb would monitor the DBUS events
> 
> 5) Get the Redfish Path from the Mapping(3) and send the Redfish event

Let me try to restate 3-5.  We already have application logic in bmcweb
that maps redfish resources to dbus interfaces, triggered by redfish
client requests (like a GET or PATCH).  The proposal here I think is to
implement the reverse - add application logic in bmcweb that maps dbus
interfaces to redfish resources, triggered by dbus signals.  I think
this is more or less what was suggested by Patrick?

> Don't we, in effect, have the mapping from Redfish to dbus by nature
> of the Redfish providers that create their content from dbus objects?

> James, Apparao: Please let me know if there is any more concern with 
> this approach.

Ratan you had 2x #2s in the flow above, which I fixed.  Please let me
know if I got it wrong.

The main concerns with this approach that I have heard are too many
signal matches and complexity of the logic implementing the reverse map.
I can definitely see how the logic will get complicated given our
current dbus data model.

One idea floating around to address these is inventing a journal
metadata scheme that is management interface agnostic.  I understand the
motivation behind that - it is much simpler for an application to slide
a single journal entry into the journal with all the metadata needed to
generate events, than it is for an application to snoop multiple signals
off dbus and pull events out of that.

But I wonder if inventing a management interface agnostic scheme for
adding events to the journal is really just inventing a new data model
for how we represent things in a server - e.g. are we just working
around our dbus data model?  Maybe we should fix it instead, so that it
isn't so difficult for applications to use it?  With that said I don't
know how to do this and I could use more concrete details on which areas
of the data model make it hard to consume signals.  Does anyone have any
ideas or examples?

Re: Redfish EventService Implementation

[Bills, Jason M]

On 6/25/2020 10:26 AM, Brad Bishop wrote:
> One idea floating around to address these is inventing a journal
> metadata scheme that is management interface agnostic.  I understand the
> motivation behind that - it is much simpler for an application to slide
> a single journal entry into the journal with all the metadata needed to
> generate events, than it is for an application to snoop multiple signals
> off dbus and pull events out of that.
> 
> But I wonder if inventing a management interface agnostic scheme for
> adding events to the journal is really just inventing a new data model
> for how we represent things in a server - e.g. are we just working
> around our dbus data model?  Maybe we should fix it instead, so that it
> isn't so difficult for applications to use it?  With that said I don't
> know how to do this and I could use more concrete details on which areas
> of the data model make it hard to consume signals.  Does anyone have any
> ideas or examples?
> 

On this, I think we may want to separate logging vs. eventing both in 
this feature discussion and in the tools we want to use.

When we were talking about logging, I think the journal made sense since 
it is designed for logging and has benefits around that usage.  However, 
I agree that it doesn't seem like the right tool for sending and 
receiving events and signals and that D-Bus sounds like the right tool 
for that.

I think I'm still a little confused at the scope.  My understanding was 
that this initial design for EventService was only for monitoring event 
messages and not resources in general.  It seems like it may not make 
sense to try to use the same tools and approach for both message 
monitoring and resource monitoring?  Do we need to treat them separately 
for now to simplify the discussion?

Re: Redfish EventService Implementation

[Ratan Gupta]

Hi Jason,james

On 6/26/20 12:25 AM, Bills, Jason M wrote:
>
>
> On 6/25/2020 10:26 AM, Brad Bishop wrote:
>> One idea floating around to address these is inventing a journal
>> metadata scheme that is management interface agnostic.  I understand the
>> motivation behind that - it is much simpler for an application to slide
>> a single journal entry into the journal with all the metadata needed to
>> generate events, than it is for an application to snoop multiple signals
>> off dbus and pull events out of that.
>>
>> But I wonder if inventing a management interface agnostic scheme for
>> adding events to the journal is really just inventing a new data model
>> for how we represent things in a server - e.g. are we just working
>> around our dbus data model?  Maybe we should fix it instead, so that it
>> isn't so difficult for applications to use it?  With that said I don't
>> know how to do this and I could use more concrete details on which areas
>> of the data model make it hard to consume signals.  Does anyone have any
>> ideas or examples?
>>
>
> On this, I think we may want to separate logging vs. eventing both in 
> this feature discussion and in the tools we want to use.
>
> When we were talking about logging, I think the journal made sense 
> since it is designed for logging and has benefits around that usage.  
> However, I agree that it doesn't seem like the right tool for sending 
> and receiving events and signals and that D-Bus sounds like the right 
> tool for that.
>
> I think I'm still a little confused at the scope.  My understanding 
> was that this initial design for EventService was only for monitoring 
> event messages and not resources in general. It seems like it may not 
> make sense to try to use the same tools and approach for both message 
> monitoring and resource monitoring? Do we need to treat them 
> separately for now to simplify the discussion?
Jason, When you say event messages? What do you mean, Do you mean to say 
"/redfish/v1/Systems/system/Logservices/eventlog"?

If yes then this should also go as Resource Event, When ever any log 
entry gets created under System Log 
(/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would 
notify to the Redfish client saying that "ResourceCreated" with the URL 
of the Resource.

After recieving this event Redfish client will do a GET request on the 
URL(retrieved as part of event) to get the content of the log.

I would be coming up with few design approaches and downside with each 
approach to take it to conclusion.

Ratan

Re: Redfish EventService Implementation

[Ratan Gupta]

On 6/26/20 12:25 AM, Bills, Jason M wrote:
>
>
> On 6/25/2020 10:26 AM, Brad Bishop wrote:
>> One idea floating around to address these is inventing a journal
>> metadata scheme that is management interface agnostic.  I understand the
>> motivation behind that - it is much simpler for an application to slide
>> a single journal entry into the journal with all the metadata needed to
>> generate events, than it is for an application to snoop multiple signals
>> off dbus and pull events out of that.
>>
>> But I wonder if inventing a management interface agnostic scheme for
>> adding events to the journal is really just inventing a new data model
>> for how we represent things in a server - e.g. are we just working
>> around our dbus data model?  Maybe we should fix it instead, so that it
>> isn't so difficult for applications to use it?  With that said I don't
>> know how to do this and I could use more concrete details on which areas
>> of the data model make it hard to consume signals.  Does anyone have any
>> ideas or examples?
>>
>
> On this, I think we may want to separate logging vs. eventing both in 
> this feature discussion and in the tools we want to use.
>
> When we were talking about logging, I think the journal made sense 
> since it is designed for logging and has benefits around that usage.  
> However, I agree that it doesn't seem like the right tool for sending 
> and receiving events and signals and that D-Bus sounds like the right 
> tool for that.
>
> I think I'm still a little confused at the scope.  My understanding 
> was that this initial design for EventService was only for monitoring 
> event messages and not resources in general. It seems like it may not 
> make sense to try to use the same tools and approach for both message 
> monitoring and resource monitoring? Do we need to treat them 
> separately for now to simplify the discussion?
Jason, When you say event messages? What do you mean, Do you mean to say 
"/redfish/v1/Systems/system/Logservices/eventlog"?

If yes then this should also go as Resource Event, When ever any log 
entry gets created under System Log 
(/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would 
notify to the Redfish client saying that "ResourceCreated" with the URL 
of the Resource.

After receiving this event Redfish client will do a GET request on the 
URL(retrieved as part of event) to get the content of the log.

This will become generic infra for all types of events.

I would be coming up with few design approaches and downside with each 
approach to take it to conclusion.

Ratan

Re: Redfish EventService Implementation

[Bills, Jason M]

On 6/29/2020 1:07 AM, Ratan Gupta wrote:
> 
> On 6/26/20 12:25 AM, Bills, Jason M wrote:
>>
>>
>> On 6/25/2020 10:26 AM, Brad Bishop wrote:
>>> One idea floating around to address these is inventing a journal
>>> metadata scheme that is management interface agnostic.  I understand the
>>> motivation behind that - it is much simpler for an application to slide
>>> a single journal entry into the journal with all the metadata needed to
>>> generate events, than it is for an application to snoop multiple signals
>>> off dbus and pull events out of that.
>>>
>>> But I wonder if inventing a management interface agnostic scheme for
>>> adding events to the journal is really just inventing a new data model
>>> for how we represent things in a server - e.g. are we just working
>>> around our dbus data model?  Maybe we should fix it instead, so that it
>>> isn't so difficult for applications to use it?  With that said I don't
>>> know how to do this and I could use more concrete details on which areas
>>> of the data model make it hard to consume signals.  Does anyone have any
>>> ideas or examples?
>>>
>>
>> On this, I think we may want to separate logging vs. eventing both in 
>> this feature discussion and in the tools we want to use.
>>
>> When we were talking about logging, I think the journal made sense 
>> since it is designed for logging and has benefits around that usage. 
>> However, I agree that it doesn't seem like the right tool for sending 
>> and receiving events and signals and that D-Bus sounds like the right 
>> tool for that.
>>
>> I think I'm still a little confused at the scope.  My understanding 
>> was that this initial design for EventService was only for monitoring 
>> event messages and not resources in general. It seems like it may not 
>> make sense to try to use the same tools and approach for both message 
>> monitoring and resource monitoring? Do we need to treat them 
>> separately for now to simplify the discussion?
> Jason, When you say event messages? What do you mean, Do you mean to say 
> "/redfish/v1/Systems/system/Logservices/eventlog"? >
> If yes then this should also go as Resource Event, When ever any log 
> entry gets created under System Log 
> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would 
> notify to the Redfish client saying that "ResourceCreated" with the URL 
> of the Resource.
Yes, new entries under 
"/redfish/v1/Systems/system/Logservices/eventlog", but I thought you 
could register for specific MessageIDs, so it's not just a generic "new 
resource" event like others would be.

> 
> After receiving this event Redfish client will do a GET request on the 
> URL(retrieved as part of event) to get the content of the log.
> 
> This will become generic infra for all types of events.
What I'm saying is I don't know if there is a good generic solution to 
cover both the EventLog and all other resources.  I believe the current 
EventService implementation was designed only for EventLog and may not 
work well for generic resource events.

> 
> I would be coming up with few design approaches and downside with each 
> approach to take it to conclusion.
Thanks!  What I'm proposing is that we clarify or possibly separate the 
discussions about EventLog vs. generic resources to avoid confusion and 
come up with the right solutions for each.

> 
> Ratan
> 

Re: Redfish EventService Implementation

[Ratan Gupta]

On 6/29/20 9:03 PM, Bills, Jason M wrote:
>
>
> On 6/29/2020 1:07 AM, Ratan Gupta wrote:
>>
>> On 6/26/20 12:25 AM, Bills, Jason M wrote:
>>>
>>>
>>> On 6/25/2020 10:26 AM, Brad Bishop wrote:
>>>> One idea floating around to address these is inventing a journal
>>>> metadata scheme that is management interface agnostic.  I 
>>>> understand the
>>>> motivation behind that - it is much simpler for an application to 
>>>> slide
>>>> a single journal entry into the journal with all the metadata 
>>>> needed to
>>>> generate events, than it is for an application to snoop multiple 
>>>> signals
>>>> off dbus and pull events out of that.
>>>>
>>>> But I wonder if inventing a management interface agnostic scheme for
>>>> adding events to the journal is really just inventing a new data model
>>>> for how we represent things in a server - e.g. are we just working
>>>> around our dbus data model?  Maybe we should fix it instead, so 
>>>> that it
>>>> isn't so difficult for applications to use it?  With that said I don't
>>>> know how to do this and I could use more concrete details on which 
>>>> areas
>>>> of the data model make it hard to consume signals.  Does anyone 
>>>> have any
>>>> ideas or examples?
>>>>
>>>
>>> On this, I think we may want to separate logging vs. eventing both 
>>> in this feature discussion and in the tools we want to use.
>>>
>>> When we were talking about logging, I think the journal made sense 
>>> since it is designed for logging and has benefits around that usage. 
>>> However, I agree that it doesn't seem like the right tool for 
>>> sending and receiving events and signals and that D-Bus sounds like 
>>> the right tool for that.
>>>
>>> I think I'm still a little confused at the scope.  My understanding 
>>> was that this initial design for EventService was only for 
>>> monitoring event messages and not resources in general. It seems 
>>> like it may not make sense to try to use the same tools and approach 
>>> for both message monitoring and resource monitoring? Do we need to 
>>> treat them separately for now to simplify the discussion?
>> Jason, When you say event messages? What do you mean, Do you mean to 
>> say "/redfish/v1/Systems/system/Logservices/eventlog"? >
>> If yes then this should also go as Resource Event, When ever any log 
>> entry gets created under System Log 
>> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would 
>> notify to the Redfish client saying that "ResourceCreated" with the 
>> URL of the Resource.
> Yes, new entries under 
> "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you 
> could register for specific MessageIDs, so it's not just a generic 
> "new resource" event like others would be.

Can we register for MessageID? I thought client can register for whole 
registry not a specific Message ID.

>
>
>>
>> After receiving this event Redfish client will do a GET request on 
>> the URL(retrieved as part of event) to get the content of the log.
>>
>> This will become generic infra for all types of events.
> What I'm saying is I don't know if there is a good generic solution to 
> cover both the EventLog and all other resources.  I believe the 
> current EventService implementation was designed only for EventLog and 
> may not work well for generic resource events.

Can you get me the example payload for EventLog which is going to be 
sent with the current design? I am not sure how the eventlog and other 
resources are different.

For eventLogs also we have the associated D-bus 
objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc)

>
>>
>> I would be coming up with few design approaches and downside with 
>> each approach to take it to conclusion.
> Thanks!  What I'm proposing is that we clarify or possibly separate 
> the discussions about EventLog vs. generic resources to avoid 
> confusion and come up with the right solutions for each.
>
>>
>> Ratan
>>

Re: Redfish EventService Implementation

[Bills, Jason M]

On 7/3/2020 3:15 AM, Ratan Gupta wrote:
>>>> I think I'm still a little confused at the scope.  My understanding 
>>>> was that this initial design for EventService was only for 
>>>> monitoring event messages and not resources in general. It seems 
>>>> like it may not make sense to try to use the same tools and approach 
>>>> for both message monitoring and resource monitoring? Do we need to 
>>>> treat them separately for now to simplify the discussion?
>>> Jason, When you say event messages? What do you mean, Do you mean to 
>>> say "/redfish/v1/Systems/system/Logservices/eventlog"? >
>>> If yes then this should also go as Resource Event, When ever any log 
>>> entry gets created under System Log 
>>> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would 
>>> notify to the Redfish client saying that "ResourceCreated" with the 
>>> URL of the Resource.
>> Yes, new entries under 
>> "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you 
>> could register for specific MessageIDs, so it's not just a generic 
>> "new resource" event like others would be.
> 
> Can we register for MessageID? I thought client can register for whole 
> registry not a specific Message ID.
> 
I don't really know.  I thought that's what the current implementation 
allowed, but I don't know for sure if it can or should.

>>
>>
>>>
>>> After receiving this event Redfish client will do a GET request on 
>>> the URL(retrieved as part of event) to get the content of the log.
>>>
>>> This will become generic infra for all types of events.
>> What I'm saying is I don't know if there is a good generic solution to 
>> cover both the EventLog and all other resources.  I believe the 
>> current EventService implementation was designed only for EventLog and 
>> may not work well for generic resource events.
> 
> Can you get me the example payload for EventLog which is going to be 
> sent with the current design? I am not sure how the eventlog and other 
> resources are different.
> 
This is based on the assumption that for a LogService, you can register 
for a MessageId.  If this is not possible, then they might be treated 
the same.

> For eventLogs also we have the associated D-bus 
> objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc)
> 
For Intel platforms, we don't use /xyz/openbmc_project/logging, so we 
don't have D-Bus objects associated with each EventLog LogEntry.  We use 
rsyslog to create a file that contains many LogEntries.

However, as an unrelated side-thought: linking logging to 
/xyz/openbmc_project/dump made me wonder if there is a possible solution 
to the logging issue if we treat /xyz/openbmc_project/logging like 
/xyz/openbmc_project/dump and place a pointer to the log in the D-Bus 
object instead of the log itself?

>>
>>>
>>> I would be coming up with few design approaches and downside with 
>>> each approach to take it to conclusion.
>> Thanks!  What I'm proposing is that we clarify or possibly separate 
>> the discussions about EventLog vs. generic resources to avoid 
>> confusion and come up with the right solutions for each.
>>
>>>
>>> Ratan
>>>

Re: Redfish EventService Implementation

[Ratan Gupta]

Hi James,

As you asked for the examples in the 
thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html), 
I have created the mapping at the following location.

https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2

1) Map1: Mapping between redfish resources to Dbus resources
2) Map2: Mapping between redfish resource types to the ineterested Dbus 
interfaces
3) Map3: Mapping between Dbus resources to redfish resources


I tried to cover the following scenario in the above mapping.

* Redfish resource is mapped to multiple Dbus Resources
* Redfish Property is mapped to single Dbus property
* Redfish Property(complex property) is mapped to multiple dbus property.
* Same type of Redfish Resources are mapped to different Dbus Resources
* Redfish node url having multiple regex : Yet to take a look on this.


Flow would be like as below

=> In bmcweb each Redfish node represents to a Redfish Resource.
=> Each node will be having it's own mapping between Redfish properties 
and the Dbus Resources.

=> Some code on bmcweb will walkthrough on each node during startup , 
get this mapping from each node and generate
    two mappings
    1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and
    2) mapping between Resource Types to the interested Dbus 
interfaces(MAP2)

=> To start with we will support few resource types and then scale it up 
as needed.

=> Map2 would be used when the Redfish client subscribe for the 
ResourceType to get the Dbus mappings.

=> Map3 would be used when the Dbus signal gets generated and need the 
Redfish mappings.

=> Once we have all thsese mapping gets generated and loaded into the 
memory, bmcweb would start listening
    for the interfaces listed in map2.

=> Once any Dbus signal gets generated map3 would be used to get the 
Redfish mapping.

Please let me know if you have any concerns with this approach.

Ratan

On 7/7/20 3:00 AM, Bills, Jason M wrote:
>
>
> On 7/3/2020 3:15 AM, Ratan Gupta wrote:
>>>>> I think I'm still a little confused at the scope.  My 
>>>>> understanding was that this initial design for EventService was 
>>>>> only for monitoring event messages and not resources in general. 
>>>>> It seems like it may not make sense to try to use the same tools 
>>>>> and approach for both message monitoring and resource monitoring? 
>>>>> Do we need to treat them separately for now to simplify the 
>>>>> discussion?
>>>> Jason, When you say event messages? What do you mean, Do you mean 
>>>> to say "/redfish/v1/Systems/system/Logservices/eventlog"? >
>>>> If yes then this should also go as Resource Event, When ever any 
>>>> log entry gets created under System Log 
>>>> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC 
>>>> would notify to the Redfish client saying that "ResourceCreated" 
>>>> with the URL of the Resource.
>>> Yes, new entries under 
>>> "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you 
>>> could register for specific MessageIDs, so it's not just a generic 
>>> "new resource" event like others would be.
>>
>> Can we register for MessageID? I thought client can register for 
>> whole registry not a specific Message ID.
>>
> I don't really know.  I thought that's what the current implementation 
> allowed, but I don't know for sure if it can or should.
>
>>>
>>>
>>>>
>>>> After receiving this event Redfish client will do a GET request on 
>>>> the URL(retrieved as part of event) to get the content of the log.
>>>>
>>>> This will become generic infra for all types of events.
>>> What I'm saying is I don't know if there is a good generic solution 
>>> to cover both the EventLog and all other resources. I believe the 
>>> current EventService implementation was designed only for EventLog 
>>> and may not work well for generic resource events.
>>
>> Can you get me the example payload for EventLog which is going to be 
>> sent with the current design? I am not sure how the eventlog and 
>> other resources are different.
>>
> This is based on the assumption that for a LogService, you can 
> register for a MessageId.  If this is not possible, then they might be 
> treated the same.
>
>> For eventLogs also we have the associated D-bus 
>> objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc)
>>
> For Intel platforms, we don't use /xyz/openbmc_project/logging, so we 
> don't have D-Bus objects associated with each EventLog LogEntry.  We 
> use rsyslog to create a file that contains many LogEntries.
>
> However, as an unrelated side-thought: linking logging to 
> /xyz/openbmc_project/dump made me wonder if there is a possible 
> solution to the logging issue if we treat /xyz/openbmc_project/logging 
> like /xyz/openbmc_project/dump and place a pointer to the log in the 
> D-Bus object instead of the log itself?
>
>>>
>>>>
>>>> I would be coming up with few design approaches and downside with 
>>>> each approach to take it to conclusion.
>>> Thanks!  What I'm proposing is that we clarify or possibly separate 
>>> the discussions about EventLog vs. generic resources to avoid 
>>> confusion and come up with the right solutions for each.
>>>
>>>>
>>>> Ratan
>>>>

Re: Redfish EventService Implementation

[James Feist]

On 7/12/2020 11:32 PM, Ratan Gupta wrote:
> Hi James,
> 
> As you asked for the examples in the 
> thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html), I 
> have created the mapping at the following location.
> 
> https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2

Thanks for the examples.

> 
> 1) Map1: Mapping between redfish resources to Dbus resources

How does this work when the mapping isn't 1:1 vs D-Bus? Most of the time 
the enums to not match the d-bus enum, or take multiple d-bus interface 
to distinguish what the value should be. Also how does this work for 
discovered things, like multiple chassis?


> 2) Map2: Mapping between redfish resource types to the ineterested Dbus 
> interfaces
> 3) Map3: Mapping between Dbus resources to redfish resources
> 
> 
> I tried to cover the following scenario in the above mapping.
> 
> * Redfish resource is mapped to multiple Dbus Resources
> * Redfish Property is mapped to single Dbus property
> * Redfish Property(complex property) is mapped to multiple dbus property.
> * Same type of Redfish Resources are mapped to different Dbus Resources
> * Redfish node url having multiple regex : Yet to take a look on this.
> >
> Flow would be like as below
> 
> => In bmcweb each Redfish node represents to a Redfish Resource.
> => Each node will be having it's own mapping between Redfish properties 
> and the Dbus Resources.
> 
> => Some code on bmcweb will walkthrough on each node during startup , 
> get this mapping from each node and generate
>     two mappings
>     1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and
>     2) mapping between Resource Types to the interested Dbus 
> interfaces(MAP2)
> 
> => To start with we will support few resource types and then scale it up 
> as needed.

I think we need an idea of what the final solution will look like for 
more complicated properties, or we'll be creating something that isn't 
future proof.

> 
> => Map2 would be used when the Redfish client subscribe for the 
> ResourceType to get the Dbus mappings.
> 
> => Map3 would be used when the Dbus signal gets generated and need the 
> Redfish mappings.

Why can't these be the same mapping? I think having 3 different maps 
makes this very confusing. I also think this is attempting to generalize 
the problem too early. If you look at the Redfish code to determine some 
of the more complicated properties, sometimes it takes quite a bit of 
logic. That logic also would possibly be nice to reuse. Maybe we can 
take your idea of a match with a callback to some of the already 
existing property parsing? Obviously it would need some cleaning up, but 
I could see something with a map of schemas/properties to function 
pointers for property parsing.

> 
> => Once we have all thsese mapping gets generated and loaded into the 
> memory, bmcweb would start listening
>     for the interfaces listed in map2.

Why aren't these compiled in? I don't see why they need to be loaded, 
could just be in code. Also, they shouldn't be added as matches until 
there is a subscriber, or we'll add many unneeded matches.

> 
> => Once any Dbus signal gets generated map3 would be used to get the 
> Redfish mapping.
> 
> Please let me know if you have any concerns with this approach.
> 
> Ratan

Re: Redfish EventService Implementation

[Ratan Gupta]

Hi James,

On 7/15/20 2:38 AM, James Feist wrote:
> On 7/12/2020 11:32 PM, Ratan Gupta wrote:
>> Hi James,
>>
>> As you asked for the examples in the 
>> thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html), 
>> I have created the mapping at the following location.
>>
>> https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2
>
> Thanks for the examples.
>
>>
>> 1) Map1: Mapping between redfish resources to Dbus resources
>
> How does this work when the mapping isn't 1:1 vs D-Bus? Most of the 
> time the enums to not match the d-bus enum, or take multiple d-bus 
> interface to distinguish what the value should be. Also how does this 
> work for discovered things, like multiple chassis?

I have not mentioned in the below use cases, but just look at the url 
https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2#file-gistfile1-txt-L77
<https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2#file-gistfile1-txt-L77>

There we are covering the resources which are getting added dynamically.
>
>
>> 2) Map2: Mapping between redfish resource types to the ineterested 
>> Dbus interfaces
>> 3) Map3: Mapping between Dbus resources to redfish resources
>>
>>
>> I tried to cover the following scenario in the above mapping.
>>
>> * Redfish resource is mapped to multiple Dbus Resources
>> * Redfish Property is mapped to single Dbus property
>> * Redfish Property(complex property) is mapped to multiple dbus 
>> property.
>> * Same type of Redfish Resources are mapped to different Dbus Resources
>> * Redfish node url having multiple regex : Yet to take a look on this.
>> >
>> Flow would be like as below
>>
>> => In bmcweb each Redfish node represents to a Redfish Resource.
>> => Each node will be having it's own mapping between Redfish 
>> properties and the Dbus Resources.
>>
>> => Some code on bmcweb will walkthrough on each node during startup , 
>> get this mapping from each node and generate
>>     two mappings
>>     1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and
>>     2) mapping between Resource Types to the interested Dbus 
>> interfaces(MAP2)
>>
>> => To start with we will support few resource types and then scale it 
>> up as needed.
>
> I think we need an idea of what the final solution will look like for 
> more complicated properties, or we'll be creating something that isn't 
> future proof.
I thought I covered the complicated one in the examples(gist link above) 
but just let me know the other redfish resource which I can take it in 
the examples.
>
>>
>> => Map2 would be used when the Redfish client subscribe for the 
>> ResourceType to get the Dbus mappings.
>>
>> => Map3 would be used when the Dbus signal gets generated and need 
>> the Redfish mappings.
>
> Why can't these be the same mapping?

To start monitoring for Dbus : We need Dbus Resource Path,

Once we get any Dbus signal, we need to send redfish event which needs 
redfish resource path hence we need reverse map

Did I get you correctly?

> I think having 3 different maps makes this very confusing. I also 
> think this is attempting to generalize the problem too early. If you 
> look at the Redfish code to determine some of the more complicated 
> properties, sometimes it takes quite a bit of logic. That logic also 
> would possibly be nice to reuse. Maybe we can take your idea of a 
> match with a callback to some of the already existing property 
> parsing? Obviously it would need some cleaning up, but I could see 
> something with a map of schemas/properties to function pointers for 
> property parsing.
>
>>
>> => Once we have all thsese mapping gets generated and loaded into the 
>> memory, bmcweb would start listening
>>     for the interfaces listed in map2.
>
> Why aren't these compiled in? I don't see why they need to be loaded, 
> could just be in code. Also, they shouldn't be added as matches until 
> there is a subscriber, or we'll add many unneeded matches.

They would be compiled in, they would be in memory(big std::map).

Agree monitoring should start only when there is a subscriber.

>
>>
>> => Once any Dbus signal gets generated map3 would be used to get the 
>> Redfish mapping.
>>
>> Please let me know if you have any concerns with this approach.
>>
>> Ratan