Showing posts with label size. Show all posts
Showing posts with label size. Show all posts

Friday, March 23, 2012

Replace or update data?

Hi,

I have products in a database each of which have varying amounts of data describing them.

Some products have variations e.g. color size, and some have price bands e.g. quantity 1-10 = $5 : 11+ = $6. This extra data is stored in seperate tables to the main products table. The rows in these other tables reference a product ID in the main table.

The question is, if an administrator were to update the product data but only change, say, the name or cost of a single product variation or the price of a single price band is it worth keeping track of exactly which item of data was changed and update that piece of data in the database or would it be better to just scrub all the data in the extra tables for the current product and re-insert all of it fresh??

Cheers,

I.From an efficiency point of view you should just update the value(s) that have changed. Inserts can be an expensive operation, particularly if there are triggers or indices to deal with.

If you're using DataSets along with a DataAdapter this will keep track of which row(s) changed, which were deleted/inserted and so call the appropriate SQL to refresh the database.|||Would you recomend cacheing the dataset in session state or viewstate?|||By caching do you mean saving the dataset for use during a postback? If so then it depends. Session state will lead to quicker response times as the cached data doesn't need to make a roundtrip to the client. But using session state will eat up your server's RAM and could make the site less responsive.

If you don't think the use of server side memory wil be an issue go with session state. Be sure to clear out the DataSet once you don't need it anymore. The .Net Framework will do this eventually but it would help to clear it out as soon as its not needed. (I assume you're not using a clustered web server which would make using session state a little trickier for complex objects like DataSets.)

If by caching you mean allowing the DataSet to be used by multiple users then you should place it into the Cache object. The Cache object is visible to all users and they share the same data.|||Yeah, for use during a postback.

Another concern I have is the cost of using a dataset update.

Upon update, a sql query is executed for each row in the dataset that's been modified/added/deleted. Would it not be better to wrap all the data in XML and send it direct to the database all in one go? Of course, doing that would mean manually checking to determine the operation required for each row.

Thanks,

WT.|||I think the DataSet update with a DataAdapter will be about as efficient as you can get, at least without doing some more coding. I haven't tried the XML route but it seems like you'd be adding the overhead of serializing the data to XML and then de-serializing it back to get it into the database.|||Thanks for your help.

I think a good reason for using viewstate for saving data inbetween postbacks is that viewstate doesn't time out.

Also, although I'll be trying this approach for product variations, I think that my original example of price bands might be better suited to deleting all the current data and replacing it.

This is because the price bands need to be consistent with one another and they also need to be complete. Imagine that whilst someone is editing a set of price bands, another user deletes all of them and gives the product a single price. If the first user then adds a new price band, upon update of the dataset, the new price band will be added and no other updates will take place. This would result in the situation of there being a single price band for the product and no price band indicating the price of 1 item. (this was deleted by the second user.)

Similarly, imagine that two different users add a new price band to a product's current set. The bands have the same lower bound but a different price. The first band is inserted but the second can't be because a unique contraint at the database forbids two bands having the same lower bound. So in this case, extra code would be needed tp recognise that an update needs to be used instead of an insert.

So in this case, I think that a 'last one in wins' approach is best to updating these price bands rather than trying to merge different sets together.

What do you think?

WT.

Saturday, February 25, 2012

reorg PK clustered index in VLDB

My data warehouse is now 4 years old and the size is about
600 GB od data.
We had only data insert no update in this duration. PKs
are all clustered indexes and in chronological order.
Some of tables are as large as 100 GB.
I wanted to check if the PKs are in good figure and ran
DBCC showcontig against many of large tables and 80 % had
bad rate for scan density , such as 50 %, 30 %.
I may need to reorganise PK.
I BOL it says comparing the values of Extent Switches and
Extents Scanned is a way to know how much fragmented. But
it says this method does not work if the index spans
multiple files. I presume all VLDB exploit multiple files
for one table in order to gain physical disk I/O.
My question: how can I check fragmentation rate of my
large tables which span multiple files (up to 4 to 6
files)?
What is the best way to reorganise clustered index which
are PK ? I have to drop all FK in order to reorganise PK,
don't I !
I hope to hear your idea!!!When you say you presume your db spans multiple files, does the database use
more than one file other than the MDF? DBCC DBreindex on the clustered key
should reindex your data tables and automatically reindex your other
nonclustered indexes.
Some links:
http://www.microsoft.com/technet/co...ql/sql0326.mspx
http://www.microsoft.com/technet/co...ql/sql1014.mspx
http://www.sqlservercentral.com/scr...butions/721.asp
Ray Higdon MCSE, MCDBA, CCNA
--
"didi" <anonymous@.discussions.microsoft.com> wrote in message
news:140c01c40b37$dd2c95d0$3501280a@.phx.gbl...
> My data warehouse is now 4 years old and the size is about
> 600 GB od data.
> We had only data insert no update in this duration. PKs
> are all clustered indexes and in chronological order.
> Some of tables are as large as 100 GB.
> I wanted to check if the PKs are in good figure and ran
> DBCC showcontig against many of large tables and 80 % had
> bad rate for scan density , such as 50 %, 30 %.
> I may need to reorganise PK.
> I BOL it says comparing the values of Extent Switches and
> Extents Scanned is a way to know how much fragmented. But
> it says this method does not work if the index spans
> multiple files. I presume all VLDB exploit multiple files
> for one table in order to gain physical disk I/O.
> My question: how can I check fragmentation rate of my
> large tables which span multiple files (up to 4 to 6
> files)?
> What is the best way to reorganise clustered index which
> are PK ? I have to drop all FK in order to reorganise PK,
> don't I !
> I hope to hear your idea!!!|||MDF file is used only for system table in all of my
databases. (especially when dealing with VLDB).
The database is over 600GB, and each table could be nearly
100GB,
Would DBreindex a good solution ?
This will copy the whole table into different location
without asking !

>--Original Message--
>When you say you presume your db spans multiple files,
does the database use
>more than one file other than the MDF? DBCC DBreindex on
the clustered key
>should reindex your data tables and automatically reindex
your other
>nonclustered indexes.
>Some links:
>http://www.microsoft.com/technet/co...chats/trans/sql
/sql0326.mspx
>http://www.microsoft.com/technet/co...chats/trans/sql
/sql1014.mspx
>http://www.sqlservercentral.com/scr...tributions/721.
asp
>--
>Ray Higdon MCSE, MCDBA, CCNA
>--
>"didi" <anonymous@.discussions.microsoft.com> wrote in
message
>news:140c01c40b37$dd2c95d0$3501280a@.phx.gbl...
about
had
and
But
files
PK,
>
>.
>|||Did those links help?
Ray Higdon MCSE, MCDBA, CCNA
--
"didi" <anonymous@.discussions.microsoft.com> wrote in message
news:148601c40b4b$a0c711b0$3a01280a@.phx.gbl...
> MDF file is used only for system table in all of my
> databases. (especially when dealing with VLDB).
> The database is over 600GB, and each table could be nearly
> 100GB,
> Would DBreindex a good solution ?
> This will copy the whole table into different location
> without asking !
>
> does the database use
> the clustered key
> your other
> /sql0326.mspx
> /sql1014.mspx
> asp
> message
> about
> had
> and
> But
> files
> PK,|||Links were very good! Thank you very much!
Especially Index Defrag Best Practices.
So, according to the article I should use fragmentation
level by logical scan fragmentation.
Still I am not very sure about using DBCC INDEXDEFRAG.
Because when a table is 100GB, and do this operation, how
large the log should be allocated ? 200 GB, 300 GB ?
Usually for copying data it takes about 2.5 times of data
size consumed in log before the data is inserted into.
Would DBCC INDEXDEFRAG be a best way in VLDB environment ?

>--Original Message--
>Did those links help?
>--
>Ray Higdon MCSE, MCDBA, CCNA
>--
>"didi" <anonymous@.discussions.microsoft.com> wrote in
message
>news:148601c40b4b$a0c711b0$3a01280a@.phx.gbl...
nearly
on
reindex
>http://www.microsoft.com/technet/co...chats/trans/sql
>http://www.microsoft.com/technet/co...chats/trans/sql
>http://www.sqlservercentral.com/scr...tributions/721.
PKs
ran
which
>
>.
>|||Depends on the needed uptime of your DB, you can write scripts to defrag in
chunks. Here is an example of using dbreindex (you can alter to use index
defrag) and backing up the log when needed, think I got this from MVP Andrew
Kelly but not 100% sure:
-- Reindexing the tables --
SET NOCOUNT ON
DECLARE @.TableName VARCHAR(100), @.Counter INT
SET @.Counter = 1
DECLARE curTables CURSOR STATIC LOCAL
FOR
SELECT Table_Name
FROM Information_Schema.Tables
WHERE Table_Type = 'BASE TABLE'
OPEN curTables
FETCH NEXT FROM curTables INTO @.TableName
SET @.TableName = RTRIM(@.TableName)
WHILE @.@.FETCH_STATUS = 0
BEGIN
SELECT 'Reindexing ' + @.TableName
DBCC DBREINDEX (@.TableName)
SET @.Counter = @.Counter + 1
-- Backup the Log every so often so as not to fill the log
IF @.Counter % 10 = 0
BEGIN
BACKUP LOG [Presents] TO [DD_Presents_Log] WITH NOINIT , NOUNLOAD
,
NAME = N'Presents Log Backup', NOSKIP , STATS = 10,
NOFORMAT
END
FETCH NEXT FROM curTables INTO @.TableName
END
CLOSE curTables
DEALLOCATE curTables
Ray Higdon MCSE, MCDBA, CCNA
--
"didi" <anonymous@.discussions.microsoft.com> wrote in message
news:159401c40c21$2fc43b60$3a01280a@.phx.gbl...
> Links were very good! Thank you very much!
> Especially Index Defrag Best Practices.
> So, according to the article I should use fragmentation
> level by logical scan fragmentation.
> Still I am not very sure about using DBCC INDEXDEFRAG.
> Because when a table is 100GB, and do this operation, how
> large the log should be allocated ? 200 GB, 300 GB ?
> Usually for copying data it takes about 2.5 times of data
> size consumed in log before the data is inserted into.
> Would DBCC INDEXDEFRAG be a best way in VLDB environment ?
>
> message
> nearly
> on
> reindex
> PKs
> ran
> which

Reorg indexes w/o log growing - how?

Hello:
is there a way to reorg indexes without the log growing to the size of the
whole db and more?
regards1. backup the log frequently
2. Change DB to "Simple Recovery" Mode
Greg Jackson
PDX, Oregon|||You could try DBCC INDEXDEFRAG instead of DBREINDEX. Sometimes, it will
produce less log records. But the mileage does vary. Also, you can consider
bulk logged recovery mode.
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
"Vadim Rapp" <vr@.myrealbox.nospam.com> wrote in message
news:O%23k6hXGEEHA.2576@.TK2MSFTNGP11.phx.gbl...
> Hello:
> is there a way to reorg indexes without the log growing to the size of the
> whole db and more?
> regards
>

reorder size of secondary data files?

Hello,
I have a database which secondary files are splitted into
4 pieces. Each Piece on a severall harddisk (18GB).
Now i have the following situation:
file1(.ndf)=5GB
file2(.ndf)=11GB
file3(.ndf)=17GB
file4(.ndf)=7GB
Is there a possibility to balance the filesize?
The harddisk for file3 is at the limit and doesn't spend
any performance!!!
10GB for Each file and a balanced growth would be nice!!!
Thanks a lot
CharibertYou can shrink the bigger files so that data from it gets "pushed" over to the smaller file (I
suggest you pre-increase the size of the smaller files before this).
Above is only way I can think of, unless you want to go through unload/import route.
--
Tibor Karaszi, SQL Server MVP
Archive at: http://groups.google.com/groups?oi=djq&as ugroup=microsoft.public.sqlserver
"Charibert Greif" <Greif@.gob.de> wrote in message news:44f401c376af$0ebe9030$a301280a@.phx.gbl...
> Hello,
> I have a database which secondary files are splitted into
> 4 pieces. Each Piece on a severall harddisk (18GB).
> Now i have the following situation:
> file1(.ndf)=5GB
> file2(.ndf)=11GB
> file3(.ndf)=17GB
> file4(.ndf)=7GB
> Is there a possibility to balance the filesize?
> The harddisk for file3 is at the limit and doesn't spend
> any performance!!!
> 10GB for Each file and a balanced growth would be nice!!!
> Thanks a lot
> Charibert|||Have you tried DBCC SHRINKFILE
For more details please refer to BOL
"Charibert Greif" <Greif@.gob.de> wrote in message
news:44f401c376af$0ebe9030$a301280a@.phx.gbl...
> Hello,
> I have a database which secondary files are splitted into
> 4 pieces. Each Piece on a severall harddisk (18GB).
> Now i have the following situation:
> file1(.ndf)=5GB
> file2(.ndf)=11GB
> file3(.ndf)=17GB
> file4(.ndf)=7GB
> Is there a possibility to balance the filesize?
> The harddisk for file3 is at the limit and doesn't spend
> any performance!!!
> 10GB for Each file and a balanced growth would be nice!!!
> Thanks a lot
> Charibert|||Hello,
I have noticed that SQL server tries to automatically
balance infill of the files in filegroup.
Recently I've added one file to each filegroup in
my DB but existed files weren't filled completely and
now I see that server insert new data into the new
files only and execution of INDEXDEFRAG is actively
moving data into new files!
So, in your case, it may be useful to increase size of
some files, run some reorganisation work (rebuilding
indexes is the best way), and later shrink files which
will get free space.
> I have a database which secondary files are splitted into
> 4 pieces. Each Piece on a severall harddisk (18GB).
> Now i have the following situation:
> file1(.ndf)=5GB
> file2(.ndf)=11GB
> file3(.ndf)=17GB
> file4(.ndf)=7GB
> Is there a possibility to balance the filesize?
> The harddisk for file3 is at the limit and doesn't spend
> any performance!!!
> 10GB for Each file and a balanced growth would be nice!!!
Serge Shakhov

Monday, February 20, 2012

Rendering: PDF and Landscape

I have a (very simple) report where I specifiy 11 X 8.5 as the paper size.
The report renders in HTML correctly in Landscape mode. When I select PDF
and Export the report is reformated and rendered as 8.5 X 11 (portait mode).
How can I fix this?
thanks
dlr"blackshirt" <blackshirt@.discussions.microsoft.com> wrote in message news:<A9031067-D190-44C0-98A6-C0155B65ACE9@.microsoft.com>...
> Dennis,
> Did you ever find the answer. I'm having the same issu.
> Thanks
> "Dennis Redfield" wrote:
> > I have a (very simple) report where I specifiy 11 X 8.5 as the paper size.
> > The report renders in HTML correctly in Landscape mode. When I select PDF
> > and Export the report is reformated and rendered as 8.5 X 11 (portait mode).
> > How can I fix this?
> >
> > thanks
> >
> > dlr
> >
> >
> >
try this:
from the main menu,
1. go to Report->Report Properties...
2. on the 'Report Properties' window go to the 'Layout' tab
3. then proceed to the Page width and Page Height controls. they have
default values of 8.5 for the width and 11 for the height.
4. Reverse the values for the width to be 11in and the height to be
8.5in
try it and let me know how it goes.
Good luck!|||I was going from a published report and switching from portrait to landscape
mode and the report was snap shot cached. In the end I deleted the report
(using RM) and re-deployed the report (from VS). after that I was fine.
dlr
"A Gutie" <fiututor@.yahoo.com> wrote in message
news:eca873f7.0409160622.58362840@.posting.google.com...
> "blackshirt" <blackshirt@.discussions.microsoft.com> wrote in message
news:<A9031067-D190-44C0-98A6-C0155B65ACE9@.microsoft.com>...
> > Dennis,
> > Did you ever find the answer. I'm having the same issu.
> >
> > Thanks
> >
> > "Dennis Redfield" wrote:
> >
> > > I have a (very simple) report where I specifiy 11 X 8.5 as the paper
size.
> > > The report renders in HTML correctly in Landscape mode. When I select
PDF
> > > and Export the report is reformated and rendered as 8.5 X 11 (portait
mode).
> > > How can I fix this?
> > >
> > > thanks
> > >
> > > dlr
> > >
> > >
> > >
> try this:
> from the main menu,
> 1. go to Report->Report Properties...
> 2. on the 'Report Properties' window go to the 'Layout' tab
> 3. then proceed to the Page width and Page Height controls. they have
> default values of 8.5 for the width and 11 for the height.
> 4. Reverse the values for the width to be 11in and the height to be
> 8.5in
> try it and let me know how it goes.
> Good luck!