db2ReadLog API looks for pruned log files.

Hi Michael,

Two things I need clarification before you close this IBM ticket.

1. Does IBM have any internal enhancement request, as you have mentioned in one of your past reply that the issue could be considered as an enhancement to API interface?

2. have you updated the IBM doc as you suggested in one of your past reply?

Thanks.

No update for 3 months, closing. If anything changes please re-open with details, thanks.

Hi. For the given text:

"To begin reading log records, the caller can specify DB2READLOG_READ or DB2READLOG_READ_SINGLE. The LRI passed by the caller does not necessarily exist on a log steam, in which case the database manager returns the next log record."

I think you might be mis-intrepreting what this means and probably we should add more clarification.

A LRI may not be found on a log stream for two reasons:

1. The log file that it should be found in cannot be found/accessed.
2. The LRI was never generated on the log stream. A LRI is made up of a LFS and LSN. A LFS/LSN tuple uniquely identifies a log record in pureScale systems across all log streams. So if you provide a LRI with a LFS/LSN it naturally will only exist on one of the log streams. For that log stream you will get back the log record. On the other log streams you will get back the first log record beyond the LFS/LSN provided. But, the log files that hold these log records need to be available otherwise the scan will fail.

The text you quote refers to (2). We need to know the difference between data missing and data not generated. If in our search for the starting point if data is missing we will report an error as you are seeing. Consider the log file with the LRI is in the archives. At the time of the request the archives are unavailable. So now based on your request we would say oh well and just return back data from the oldest log file we could find. Doing that for the majority of the db2ReadLog API consumers would result in data corruption.

So I will look to get our documentation updated to be more clear.

But, your request can be considered an enhancement to the API interface. We could keep track of the last log file pruned and consider it as a lower bound for queries. If any request needs to go below this lower bound we can either (a) quietly ignore the missing log and return back data from the oldest file we found or (b) provide a new option in the API that given (a) says it is ok to ignore missing log below this lower bound,

We would prefer that you work with IBM to see if there is a way we can alter your application to avoid such a catch all LRI being used. Most consumers of the API use the query capability to find out what LRI to use and do not rely on a generic LRI input. This follows in line with our documentation that states: "To use the Asynchronous Log Reader (db2ReadLog API), first query the database log for a valid starting LRI". If you cannot do this, we would like to know why.


Thanks.

Hi,

Let me explain the issue that I am facing with db2ReadLog api.

The IBM doc

https://www.ibm.com/support/knowledgecenter/en/SSEPGG_11.1.0/com.ibm.db2.luw.apdv.api.doc/doc/r0001673.html

says

"To begin reading log records, the caller can specify DB2READLOG_READ or DB2READLOG_READ_SINGLE. The LRI passed by the caller does not necessarily exist on a log steam, in which case the database manager returns the next log record."

as per the above statement, if I want to start reading the log records(starting from the first valid record among the logs), I just need to call db2ReadLog api with DB2READLOG_READ /DB2READLOG_READ_SINGLE with some arbitrary LRI value.

Since I do not know the LRI of the first valid db2 record, I pass startLSN as '0x1' and endLSN as 0xFFFFFFFF along with DB2READLOG_READ /DB2READLOG_READ_SINGLE as arguments to db2ReadLog api.

Now there are two scenarios:

suppose at any particular point of time, My system have logs starting from sequence#‌

S0000012.LOG till S0000030.LOG and the current active log is S0000020.LOG.

scenario 1:

I call db2readLog api with above mentioned arguments and I get first valid db2 record from

S0000012.LOG.

scenario 2:

at some point of time I feel the logs prior to active log are not needed any more and I can live without these older logs. So I prune all the logs prior to current active log with prune command as shown below

bash-4.1$ db2 PRUNE LOGFILE PRIOR TO S0000020.LOG

DB20000I The PRUNE command completed successfully.

bash-4.1$ db2 connect reset

DB20000I The SQL command completed successfully.

Now, I am left with the the logs starting from log sequence#‌ S0000020.LOG. The current active log S0000020.LOG is now the oldest log in my system.

I call db2readLog api with above mentioned arguments to get the first valid log record.

but the api fails with below error

" db2ReadLog error [SC=-1273:SQL1273N An operation reading the logs on database "QADB" cannot continue because of a missing log file "S0000019.LOG" on database partition "0" and log stream "0"]."

Here api is trying to read pruned logs file(S0000019.LOG) which are not available in the system . Since api does not get the log file, It throws the error.

Why does api look for the pruned logs?

The api should always read the available and the archived logs which are physically available in the system and returns the first valid record.

Please feel free to get back to me if more clarifications are needed.

Thanks.

How would you like something like this to work? Db2 needs to tell the difference between file not there because:

1. pruned by Db2 (what you gave).
2. user manually deleted outside of Db2 commands
3. unexpected file missing

It would not be safe for Db2 to return data without knowing which scenario not finding a file falls under. So how would you suggest Db2 know under what scenario is it safe to ignore missing files and return back the first log record that satisfies LRI request?