The GetDPI Photography Forum

Great to see you here. Join our insightful photographic forum today and start tapping into a huge wealth of photographic knowledge. Completing our simple registration process will allow you to gain access to exclusive content, add your own topics and posts, share your work and connect with other members through your own private inbox! And don’t forget to say hi!

Cataloging/Library - Huge amount of MF images

Ben Rubinstein

Active member
Hi,

We will be shooting about 20tb worth of images per year and that's just the RAW files. I need a system where I can search these images based on metadata, swiftly. Now I know that LR can do this but it's pretty much a database only. Once I've found the images I can't copy a selection over to a couple of hard drives and burn a disk of them. I need not only to be able to find them but browser style organise, copy, move, etc the files. I'm on windows not mac.

Bridge is useless, taken 3 hours to index a 1tb drive and then when I do a simple metadata search it starts indexing them all over again. And again. And again. It has a powerful search interface but the search engine is a joke.

Lightroom is far better, supremely fast, easy, etc. But once I've found the files I can't do anything other than export them, I've no access to them.

There is a widget for Win 7 which allows the windows search engine to search metadata but it was made for music files, I haven't tried it on IPTC or EXIF data though I'm going to boot into the win 7 system now to try it out.

Any ideas for a fast metadata solution which allows me browser functionality over the found files?

I'm sure a lot of you people are using LR but there must be other systems out there for big studio's, stock shooters, etc who need more than that?
 

dougpeterson

Workshop Member
This sounds like the definition of what Media Pro is meant to attack.

At this point the performance and stability (lack of bugs) is not to the point where we (Capture Integration) are giving it a resounding "green light" but if there was anybody I've run into so far that fits their target market - it's you.

If you do try it be sure to let us know how it goes.

It will be on sale for 50% off tomorrow.

Doug Peterson (e-mail Me)
__________________

Head of Technical Services, Capture Integration
Phase One Partner of the Year
Mamiya Leaf, Leica, Arca Swiss, Cambo, Profoto, LaCie, Canon, TTI, Broncolor & More

National: 877.217.9870 | Cell: 740.707.2183
Newsletter | RSS Feed
Buy Capture One 6 at 10% off
 

Bob

Administrator
Staff member
I have been using media pro now for about a month.
I so far have not used its import feature since I feed it files from C1.
My method is to first sort through my images and take all third sort images and move them to the selects folder and then using the C1 context menu, catalogue them in media pro.
There I add more elaborate information into the metadata such as model, location and genre.
so far so good, but it does have a few integration issues.
-bob
 

dougpeterson

Workshop Member
Wow but it's glacially slow to import. LR is hugely faster.
True, but IMPORT SPEED would be down in maybe the 10th or 11th slot if I listed my priorities for a tool for your problem. Your first import would be huge/long/annoying, but it's not at all so slow that importing speed down the line (day to day use) is really going to cause any issues.

IMPORT STABILITY however would be in the top 5 slots, and in our testing so far it's been hit or miss (assuming the test is a stress-test meant to find any weakness) in the versions of Media Pro released so far.

So again, let us know how it goes!
 

Stefan Steib

Active member
Beni

Aperture can do this pretty good. You can search all items with some reference words or numbers or other data like exif , iptc or size or whatever the database of aperture can index, you just throw the search criterium at it and let it go. After you have the images selected you need to export, say export and let it go.
If your machine has enough Ram (8gigs and up) and a fast Harddrive (Macpro maybe with raid) the thing runs pretty good even with Large libraries.

There is a free 30 day demo. Just try it. Be warned though- the database function is pretty mighty and takes some time especially if you have not done this before. it is also taking some learning curve to compile the metadata at input to sort it correctly. But once you are over this, it´s probably the best lowcost solution out there. if you want something better than this you have to spend about 20000 € for a dedicated image database system.

regards
Stefan
 

Ben Rubinstein

Active member
Well it took an hour to try and catalogue a 200gig folder then crashed. LR did the same folder in about 15 minutes.

I've been playing with this kind of concept. Shoot in C1 tethered to local hard drive. Render JPG's for web use. Copy RAW folder over to NAS disk array where it will be catalogued by LR and fully keyworded. If I need to find a file then I can use LR to pull it up within seconds and then use 'show in explorer' to get the file/folder or use the export tool to copy the files to a new location/s.

Only annoying thing is that the export doesn't export the C1 folder with all the adjustments data, I do wish they would just use XMP files so that it would be easier to move files with their adjustments...
 

Ben Rubinstein

Active member
Beni

Aperture can do this pretty good. You can search all items with some reference words or numbers or other data like exif , iptc or size or whatever the database of aperture can index, you just throw the search criterium at it and let it go. After you have the images selected you need to export, say export and let it go.
If your machine has enough Ram (8gigs and up) and a fast Harddrive (Macpro maybe with raid) the thing runs pretty good even with Large libraries.

There is a free 30 day demo. Just try it. Be warned though- the database function is pretty mighty and takes some time especially if you have not done this before. it is also taking some learning curve to compile the metadata at input to sort it correctly. But once you are over this, it´s probably the best lowcost solution out there. if you want something better than this you have to spend about 20000 € for a dedicated image database system.

regards
Stefan
PC I'm afraid Stefan.
 

Stefan Steib

Active member
Maybe you ask some people at the universities in Tel Aviv, there are some free Unix database servers which are used for edu purposes worldwide.
You could also ask Mr. Meyer what he would recommend.
Fact is you will need a dedicated image server for that amount of stuff. Use Unix - probably Linux. Connect with Gigabit ethernet and have some money to get a server machine with enough hairs on the chest.........

regards
Stefan
 

Ben Rubinstein

Active member
Mr Mayer suggested using extremely descriptive filenames in leu of metadata for easy searching. I can see his point makes a huge difference if you can just search filenames but I was hoping for a far neater option.
 

Ben Rubinstein

Active member
Many thanks Stefan, just trying to keep everything as simple as possible. There is a custom database being built for the website which is where all these books are going to end up, that is pretty much their final destination. I just wanted something for keeping track of the raw files.
 

Bill Caulfeild-Browne

Well-known member
I've been using Media Pro for several months now. Very temperamental at first but improvements have been made and now it functions quite well for me. (I have only kept about 20,000 images though of the 100,000 I started out with - I've become very tough editor!)

I do like the immediacy of the interface with Capture One.

Bill
 

FredBGG

Not Available
I used it when it still belonged to Microsoft.
Powerful, but buggy.
I tied using it for a project.... Book of Celebrity portraits.
Hundreds of shoots. "Only" about 25,000 images.
It could not handle that many files without crashing way to much.
If Phase can make it more stable it could be a winner.
 
For that much data, you should seriously consider a proper content management system if you highly value your content, for example Alfresco, it is intended for precisely such problems. e.g. Sesame St manage their entire (50 yr?) archive of video on 5 high end servers. It's not a packaged solution, you'll probably need professional help (I can recommend a company in Vietnam if you want).

btw, it's opensource/free unless you need support from Alfresco.

There are many benefits to this approach, I can take you though it you need. Think about this archive looking like a windows share and a web application supporting multiple people all searching simultaneously, it's a different world from a desktop app.

Paul
 
S

Shelby Lewis

Guest
Have you thought about using Photomechanic just for ingest, keywording, et al... I used it on the front end of my wedding/portrait biz for a long time. It's slim and stable and I still use it from time to time.

I'll admit to not using it intensively with MF, but I think it's a fantastic program. Very powerful as far as key-wording and importing to multiple directories simultaneously.
 
Not sure, if it will be able to deal with that amount of data, but I have been using IDImager to organise my image library. It runs on an SQL database and from my experience has no problem dealing with large amount of data. Support is great, so you can just get in touch with them to ask about your specific requirements.
 
Top