OONumerics User : |
From: Xiaowen Wang (swangken_at_[hidden])
Date: 2001-07-19 09:53:19
Michael Soliman writes:
>
> Moin Moin.
> > Michael Soliman writes:
> > > Moin Moin.
> > >
> > > Did anyone hear of a map<.,.,.,.>-implemantation, shadowing its data
> into
> > > a database [maybe by replacing the normal allocator-mechanism as to
> hand
> > > out the map's index-parameter as a parameter to the allocators.
> > The runtime complexity of a database update and a memory chunk
> > allocation is magnitude far apart, the former one typically involves
> > at least one network roundtrip unless you're using embedded
> > database plus some constraint check etc. Therefore it doesn't sound a
> > good idea to bundle them together at such fine grain even though
> > syntatically it can hide the dirty database work beneath.
> >
> > If all you need is to update the database with the data in the map,
> > why not write a helper method/class to do the database work? That way,
> > all you need from the content side is an iterator of desired key and
> > value type of an associative container, which includes not only
> Hm. I assume that, by saying "associative", You mean, that the container
> comprises a bijective function
> binding together map-elements and database-table-entries.
By "associative container" I mean the kind of container mapping a key
to a value, like std::map<int, double>. You can refer to SGI STL
documentation for precise definition of the concept.
> > map<...>, but a whole lot more. Note that you don't have to change any
> > existing container code. On the DB interface side, you can devise
> > some common interface expected of your application, and then provide
> > concrete impl for different DBMS or even other data persistence means
> > like a zip file.
> Ok. This does the job as I lined it out.
> But my initial idea was to have some fast access to database-"shadowed"
> data, rather than using ONE db-access per map<.,.,.,.>-access.
> As I heard the allocators to be intended for use on different "kinds" of
> memory (say DRAM as well as FLASH). I thought one could use this mechanism,
> to implement some sort of virtual-memory-machine.
> It could execute the map<.,.,.,.>-accesses using the DRAM, as long as the
> data can be found in the db and as long as it has been read.
> Otherwise it could execute something like a page-fault just alike any other
> virtual-memory-system, accessing the database instead of a swap-file.
> This would have the advantage, that You could perform fast arithmetics on
> this kind of a map, as it would be using the same kind of
> (compiler-optimised) source as the normal STL-map<.,.> as long as no
> allocator is called.
> In essence, it would mean to use the
> virtual-memory-machine-"hardware-acceleration" (i.e., the MMU) to decide,
> whether the data has to be read from the db.
I don't think it's in the allocator level that you can do this. The
reason is that the allocator interface never bother with the actual
content to be written into the memory chunk to be allocated, it only
cares about the size. You can see from the following exerpt from the g++-2.96 stl_alloc.h:
// __n is permitted to be 0. The C++ standard says nothing about what
// the return value is when __n == 0.
_Tp* allocate(size_type __n, const void* = 0) {
return __n != 0 ? static_cast<_Tp*>(_Alloc::allocate(__n * sizeof(_Tp)))
: 0;
}
// __p is not permitted to be a null pointer.
void deallocate(pointer __p, size_type __n)
{ _Alloc::deallocate(__p, __n * sizeof(_Tp)); }
Therefore if you insist to update DB together when updating the in
memory map<key,data,alloc>, you have to do it in the map<key,data,alloc>
level rather than the alloc level.
It's a good idea to have a clear boundary of transaction when doing
database operations, however, the interface of generic containers gives no clue
when you should do the commit. That's why I think it's not a good
design to bundle a generic container like map with the DB logic.
Also it's preferred to do batch update rather than many sporadic small
updates, and we should handle sql exceptions.
The solution I mentioned in the last email can be explained as
following code:
std::map<int, double>::iterator beg = theMap.begin(), end =
theMap.end();
DBHelper helper(dbURL, usrnm, passwd); // the helper enacpsulate DB
related logic
// batch update
try {
helper.begin(); // clear out previous batch sql commands.
while(beg != end) {
// add one sql command to the sql commands batch
helper.addCheckin(beg->first, beg->second);
++beg;
}
helper.commit(); // batch update
} catch (DB related exception) {
helper.rollback();
}
As you see here, it's the iterator that are represeting the container
to get involved with the DB operations, you can subsitute the iterator
with other type: std::hashmap<int, double>:iterator,
std::multimap<int, double>::iterator etc, and the code runs
without change, because all these containers are associative container.
In short, DB is content persistence, while allocator is memory
management, the latter one is living in a much lower level API world.
Cheers
XW
> --------------------- Object Oriented Numerics List --------------------------
> * To subscribe/unsubscribe: use the handy web form at
> http://oonumerics.org/oon/
> * If this doesn't work, please send a note to owner-oon-list_at_[hidden]
--------------------- Object Oriented Numerics List --------------------------
* To subscribe/unsubscribe: use the handy web form at
http://oonumerics.org/oon/
* If this doesn't work, please send a note to owner-oon-list_at_[hidden]