CSV export directly from cursor, not via grid?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

CSV export directly from cursor, not via grid?

Kjell Rilbe-3
Hi,

I need to export a large dataset (almost 2 million records), and it
seems not to work well in FlameRobin, because after having fetched all
records into the grid, the Grid menu takes forever to open in order to
execute the "Save as CSV" command.

Would it be possible to add the option to save to CSV directly from the
SQL command/cursor, without going via the grid?

I stumble on this problem now and again, so it's not just a temporary
need for me.

If not, I guess I'll have to write my own export tool or find some third
party export tool. Recommendations welcome. Small feature set,
simplicity and free/low price more important than advanced features and
flashy GUI. :-)

Regards,
Kjell

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
Flamerobin-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/flamerobin-devel
Reply | Threaded
Open this post in threaded view
|

Re: CSV export directly from cursor, not via grid?

Milan Babuskov-4
On Thu, Apr 17, 2014 at 11:07 AM, Kjell Rilbe <[hidden email]> wrote:
> I need to export a large dataset (almost 2 million records), and it
> seems not to work well in FlameRobin, because after having fetched all
> records into the grid, the Grid menu takes forever to open in order to
> execute the "Save as CSV" command.

It loads everything into RAM and probably uses swap space.

> Would it be possible to add the option to save to CSV directly from the
> SQL command/cursor, without going via the grid?

Not with FR.

> I stumble on this problem now and again, so it's not just a temporary
> need for me.
>
> If not, I guess I'll have to write my own export tool or find some third
> party export tool. Recommendations welcome.

You can use my FBExport tool. It's bundled with FBCopy, works on
Windows and Linux:

http://www.firebirdfaq.org/fbcopy.php

And it's open source, so you can modify it. Manual is here:

http://fbexport.sourceforge.net/manual.html

> more important than advanced features and flashy GUI. :-)

No GUI. It's a command-line tool :)

--
Milan Babuskov
http://www.guacosoft.com

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
Flamerobin-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/flamerobin-devel
Reply | Threaded
Open this post in threaded view
|

Re: CSV export directly from cursor, not via grid?

Milan Babuskov-4
On Fri, Apr 18, 2014 at 11:56 AM, Milan Babuskov
<[hidden email]> wrote:
> http://www.firebirdfaq.org/fbcopy.php

Same download link, but correct tool features:

http://www.firebirdfaq.org/fbexport.php

--
Milan Babuskov
http://www.guacosoft.com

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
Flamerobin-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/flamerobin-devel
Reply | Threaded
Open this post in threaded view
|

Re: CSV export directly from cursor, not via grid?

Kjell Rilbe-3
In reply to this post by Milan Babuskov-4
Milan Babuskov skriver:
> On Thu, Apr 17, 2014 at 11:07 AM, Kjell Rilbe <[hidden email]> wrote:
>> I need to export a large dataset (almost 2 million records), and it
>> seems not to work well in FlameRobin, because after having fetched all
>> records into the grid, the Grid menu takes forever to open in order to
>> execute the "Save as CSV" command.
> It loads everything into RAM and probably uses swap space.

The grid itself works - I can load the entire dataset into the grid. But
after that, I can't open the Grid menu to choose the Export command...

>> I stumble on this problem now and again, so it's not just a temporary
>> need for me.
>>
>> If not, I guess I'll have to write my own export tool or find some third
>> party export tool. Recommendations welcome.
> You can use my FBExport tool. It's bundled with FBCopy, works on
> Windows and Linux:
>
> http://www.firebirdfaq.org/fbcopy.php
>
> And it's open source, so you can modify it. Manual is here:
>
> http://fbexport.sourceforge.net/manual.html

Having a look at it now. I need to quote my identifiers. Any luck?

Also, the query is 50 lines long. Can the tool get the SQL from a file?

And my identifiers contain Swedish letters åäö...?

>> more important than advanced features and flashy GUI. :-)
> No GUI. It's a command-line tool :)
>

Which is both good and bad. :-)

Regards,
Kjell

--
------------------------------
Kjell Rilbe
DataDIA AB
E-post: [hidden email]
Telefon: 08-761 06 55
Mobil: 0733-44 24 64


------------------------------------------------------------------------------
Start Your Social Network Today - Download eXo Platform
Build your Enterprise Intranet with eXo Platform Software
Java Based Open Source Intranet - Social, Extensible, Cloud Ready
Get Started Now And Turn Your Intranet Into A Collaboration Platform
http://p.sf.net/sfu/ExoPlatform
_______________________________________________
Flamerobin-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/flamerobin-devel
Reply | Threaded
Open this post in threaded view
|

Re: CSV export directly from cursor, not via grid?

Kjell Rilbe-3
In reply to this post by Milan Babuskov-4
Milan Babuskov skriver:
> You can use my FBExport tool. It's bundled with FBCopy, works on
> Windows and Linux: http://www.firebirdfaq.org/fbcopy.php And it's open
> source, so you can modify it. Manual is here:
> http://fbexport.sourceforge.net/manual.html

It fails saying it can't load fbclient.dll or gds32.dll. What might be
causing this, considering I copied the dll both to the FBExport EXE
subfolder (same as the exe) and Windows\system32?

(I created a view that "hides" all the problematic SQL, hoping that will
let me export the data with your tool in an easy way...)

Regards,
Kjell

------------------------------------------------------------------------------
Start Your Social Network Today - Download eXo Platform
Build your Enterprise Intranet with eXo Platform Software
Java Based Open Source Intranet - Social, Extensible, Cloud Ready
Get Started Now And Turn Your Intranet Into A Collaboration Platform
http://p.sf.net/sfu/ExoPlatform
_______________________________________________
Flamerobin-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/flamerobin-devel
Reply | Threaded
Open this post in threaded view
|

Re: CSV export directly from cursor, not via grid?

Milan Babuskov-4
In reply to this post by Kjell Rilbe-3
On Tue, Apr 22, 2014 at 11:47 AM, Kjell Rilbe <[hidden email]> wrote:
> Having a look at it now. I need to quote my identifiers. Any luck?

No such option yet.

> Also, the query is 50 lines long. Can the tool get the SQL from a file?

Only with -X option, not what you want. Can't you just create a batch
file/shell script with the command and run it?

> And my identifiers contain Swedish letters åäö...?

I assume you're using Windows? I did all the development and testing
on Linux, so I'm not sure if that works.

>> No GUI. It's a command-line tool :)
> Which is both good and bad. :-)

Sorry to disappoint you.

--
Milan Babuskov
http://www.guacosoft.com

------------------------------------------------------------------------------
Start Your Social Network Today - Download eXo Platform
Build your Enterprise Intranet with eXo Platform Software
Java Based Open Source Intranet - Social, Extensible, Cloud Ready
Get Started Now And Turn Your Intranet Into A Collaboration Platform
http://p.sf.net/sfu/ExoPlatform
_______________________________________________
Flamerobin-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/flamerobin-devel
Reply | Threaded
Open this post in threaded view
|

Re: CSV export directly from cursor, not via grid?

Michael Hieke
In reply to this post by Kjell Rilbe-3
On 22.04.2014 11:57, Kjell Rilbe wrote:

> It fails saying it can't load fbclient.dll or gds32.dll. What might
> be causing this, considering I copied the dll both to the FBExport
> EXE subfolder (same as the exe) and Windows\system32?

Try running it under depends.exe <http://www.dependencywalker.com>, the
tool has options to show all DLL load attempts and corresponding paths.
  Maybe just a 32 bit vs. 64 bit problem?

Thanks

--
Michael Hieke


------------------------------------------------------------------------------
Start Your Social Network Today - Download eXo Platform
Build your Enterprise Intranet with eXo Platform Software
Java Based Open Source Intranet - Social, Extensible, Cloud Ready
Get Started Now And Turn Your Intranet Into A Collaboration Platform
http://p.sf.net/sfu/ExoPlatform
_______________________________________________
Flamerobin-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/flamerobin-devel
Reply | Threaded
Open this post in threaded view
|

Re: CSV export directly from cursor, not via grid?

Kjell Rilbe-3
In reply to this post by Kjell Rilbe-3
Den 2014-04-17 11:07 skrev Kjell Rilbe såhär:

> Hi,
>
> I need to export a large dataset (almost 2 million records), and it
> seems not to work well in FlameRobin, because after having fetched all
> records into the grid, the Grid menu takes forever to open in order to
> execute the "Save as CSV" command.
>
> Would it be possible to add the option to save to CSV directly from the
> SQL command/cursor, without going via the grid?
>
> I stumble on this problem now and again, so it's not just a temporary
> need for me.
>
> If not, I guess I'll have to write my own export tool or find some third
> party export tool. Recommendations welcome. Small feature set,
> simplicity and free/low price more important than advanced features and
> flashy GUI. :-)

I ended up using an external table instead, but I had to fiddle with the
file in Excel and Notepad++ afterwards to get the desired csv format. Oh
well.

Regards,
Kjell



------------------------------------------------------------------------------
Start Your Social Network Today - Download eXo Platform
Build your Enterprise Intranet with eXo Platform Software
Java Based Open Source Intranet - Social, Extensible, Cloud Ready
Get Started Now And Turn Your Intranet Into A Collaboration Platform
http://p.sf.net/sfu/ExoPlatform
_______________________________________________
Flamerobin-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/flamerobin-devel