[bdbxml] Adding XML documents slows down when the container contains large number of documents[#14407]

[Younger Wang]

In my test of adding documents,

1. Size of the document is only 3KB.

2. Schema validation is not enabled.

3. I did not create any index yet.

4. I did try to adjust cache size. I tried both default cache size
(256KB) and 50M and they performed equally.

5. My test program is single-threaded. The program reads a template from
container A, generates a new XML document according to the template, and
saves the new document to container B. I am quite sure reading from
container A and generating a new XML document did not slow down because the
program completes instantly when the adding part was commented.

6. Transaction is enabled.

7. DB_CONFIG is:

set_lk_max_lockers 10000

set_lk_max_objects 10000

set_lk_max_locks 10000



BR

Younger Wang





_____

From: Younger Wang [mailto:younger.wang-***@public.gmane.org]
Sent: 2006Äê4ÔÂ14ÈÕ 14:06
To: xml-***@public.gmane.org
Subject: [bdbxml] Adding XML documents slows down when the container
contains large number of documents[#14407]



Hi,



When the container is empty, adding 100 documents takes 32 seconds; When the
container contains 10,000 documents, adding 100 takes 492 seconds. That is
too slow. How can I solve the problem? Thanks!



BR
Younger Wang

[Younger Wang]

Hi,



I think I have found the cause. Before adding a new document, my program
queries against the database. Since I haven¡¯t created any index, the query
takes a very long time when the database becomes large.



Now I have another question. In my XML documents, namespace is not declared.
When I add an index, which namespace shall I use? I added index like ¡°
addindex "" CPEInfo/ModelID edge-element-equality-string ¡°, but the query
plan showed the index was invalid for ¡°
collection("cpeinfo.dbxml")/CPEInfo/ModelID ¡±



BR

Younger Wang

_____

From: Younger Wang [mailto:younger.wang-***@public.gmane.org]
Sent: 2006Äê4ÔÂ17ÈÕ 12:51
To: xml-***@public.gmane.org
Subject: RE: [bdbxml] Adding XML documents slows down when the container
contains large number of documents[#14407]



In my test of adding documents,

1. Size of the document is only 3KB.

2. Schema validation is not enabled.

3. I did not create any index yet.

4. I did try to adjust cache size. I tried both default cache size
(256KB) and 50M and they performed equally.

5. My test program is single-threaded. The program reads a template from
container A, generates a new XML document according to the template, and
saves the new document to container B. I am quite sure reading from
container A and generating a new XML document did not slow down because the
program completes instantly when the adding part was commented.

6. Transaction is enabled.

7. DB_CONFIG is:

set_lk_max_lockers 10000

set_lk_max_objects 10000

set_lk_max_locks 10000



BR

Younger Wang





_____

From: Younger Wang [mailto:younger.wang-***@public.gmane.org]
Sent: 2006Äê4ÔÂ14ÈÕ 14:06
To: xml-***@public.gmane.org
Subject: [bdbxml] Adding XML documents slows down when the container
contains large number of documents[#14407]



Hi,



When the container is empty, adding 100 documents takes 32 seconds; When the
container contains 10,000 documents, adding 100 takes 492 seconds. That is
too slow. How can I solve the problem? Thanks!



BR
Younger Wang

[Younger Wang]

Hi,



I think I have found the cause. Before adding a new document, my program
queries against the database. Since I haven¡¯t created any index, the query
takes a very long time when the database becomes large.



Now I have another question. In my XML documents, namespace is not declared.
When I add an index, which namespace shall I use? I added index like ¡°
addindex "" CPEInfo/ModelID edge-element-equality-string ¡°, but the query
plan showed the index was invalid for ¡°
collection("cpeinfo.dbxml")/CPEInfo/ModelID ¡±



BR

Younger Wang





_____

From: Younger Wang [mailto:younger.wang-***@public.gmane.org]
Sent: 2006Äê4ÔÂ17ÈÕ 12:51
To: xml-***@public.gmane.org
Subject: RE: [bdbxml] Adding XML documents slows down when the container
contains large number of documents[#14407]



In my test of adding documents,

1. Size of the document is only 3KB.

2. Schema validation is not enabled.

3. I did not create any index yet.

4. I did try to adjust cache size. I tried both default cache size
(256KB) and 50M and they performed equally.

5. My test program is single-threaded. The program reads a template from
container A, generates a new XML document according to the template, and
saves the new document to container B. I am quite sure reading from
container A and generating a new XML document did not slow down because the
program completes instantly when the adding part was commented.

6. Transaction is enabled.

7. DB_CONFIG is:

set_lk_max_lockers 10000

set_lk_max_objects 10000

set_lk_max_locks 10000



BR

Younger Wang





_____

From: Younger Wang [mailto:younger.wang-***@public.gmane.org]
Sent: 2006Äê4ÔÂ14ÈÕ 14:06
To: xml-***@public.gmane.org
Subject: [bdbxml] Adding XML documents slows down when the container
contains large number of documents[#14407]



Hi,



When the container is empty, adding 100 documents takes 32 seconds; When the
container contains 10,000 documents, adding 100 takes 492 seconds. That is
too slow. How can I solve the problem? Thanks!



BR
Younger Wang

[Younger Wang]

Stupid me! I used XPath to specify the node. The right way is ¡° addindex
"" ModelID edge-element-equality-string ¡±



BR

Younger Wang



_____

From: Younger Wang [mailto:younger.wang-***@public.gmane.org]
Sent: 2006Äê4ÔÂ17ÈÕ 12:59
To: xml-***@public.gmane.org
Subject: RE: [bdbxml] Adding XML documents slows down when the container
contains large number of documents[#14407]



Hi,



I think I have found the cause. Before adding a new document, my program
queries against the database. Since I haven¡¯t created any index, the query
takes a very long time when the database becomes large.



Now I have another question. In my XML documents, namespace is not declared.
When I add an index, which namespace shall I use? I added index like ¡°
addindex "" CPEInfo/ModelID edge-element-equality-string ¡°, but the query
plan showed the index was invalid for ¡°
collection("cpeinfo.dbxml")/CPEInfo/ModelID ¡±



BR

Younger Wang





_____

From: Younger Wang [mailto:younger.wang-***@public.gmane.org]
Sent: 2006Äê4ÔÂ17ÈÕ 12:51
To: xml-***@public.gmane.org
Subject: RE: [bdbxml] Adding XML documents slows down when the container
contains large number of documents[#14407]



In my test of adding documents,

1. Size of the document is only 3KB.

2. Schema validation is not enabled.

3. I did not create any index yet.

4. I did try to adjust cache size. I tried both default cache size
(256KB) and 50M and they performed equally.

5. My test program is single-threaded. The program reads a template from
container A, generates a new XML document according to the template, and
saves the new document to container B. I am quite sure reading from
container A and generating a new XML document did not slow down because the
program completes instantly when the adding part was commented.

6. Transaction is enabled.

7. DB_CONFIG is:

set_lk_max_lockers 10000

set_lk_max_objects 10000

set_lk_max_locks 10000



BR

Younger Wang





_____

From: Younger Wang [mailto:younger.wang-***@public.gmane.org]
Sent: 2006Äê4ÔÂ14ÈÕ 14:06
To: xml-***@public.gmane.org
Subject: [bdbxml] Adding XML documents slows down when the container
contains large number of documents[#14407]



Hi,



When the container is empty, adding 100 documents takes 32 seconds; When the
container contains 10,000 documents, adding 100 takes 492 seconds. That is
too slow. How can I solve the problem? Thanks!



BR
Younger Wang