[LUGSB] text editors for 4+ GB files

Fri Nov 19 17:39:05 EST 2004

Hi All,
Can anyone recommed a good txt editor for looking at 4+ GB txt files? Vim takes for ever to open up. I'd ideally like  to be able to open a file at a given line or at the first occurance of a given pattern, ala vim. I definetly need to be able to go to the first occurance of a given pattern or a given line once the file is open. 
I have an 4-cpu alpha with 6GB of ram, so memory is not too much of an issue.

% wc output.file
 113401439 547030563 5061674108

 still waiting for vim to open up to line 100600323. By this time, I've already grepped fully through the file 3 times.

 -Lucas
>From vinay at mnl.cs.sunysb.edu Fri Nov 19 17:49:07 2004
Received: from gateway (mail@[130.245.128.9])iAJMn6WW005582
	for <lugsb at fsl.cs.sunysb.edu>; Fri, 19 Nov 2004 17:49:07 -0500
Received: from linux11.mnl.cs.sunysb.edu ([130.245.145.69] ident=mail)
	by gateway with esmtp (Exim 3.35 #1 (Debian))
	id 1CVHZY-0006P2-00
	for <lugsb at fsl.cs.sunysb.edu>; Fri, 19 Nov 2004 17:49:44 -0500
Received: from vinay (helo=localhost)
	by linux11.mnl.cs.sunysb.edu with local-esmtp (Exim 3.36 #1 (Debian))
	id 1CVHZY-0001Ps-00
	for <lugsb at fsl.cs.sunysb.edu>; Fri, 19 Nov 2004 17:49:44 -0500
Date: Fri, 19 Nov 2004 17:49:44 -0500 (EST)
From: Vinay Pai <vinay at cs.sunysb.edu>
X-X-Sender: vinay at linux11.mnl.cs.sunysb.edu
To: Linux Users Group at Stony Brook <lugsb at fsl.cs.sunysb.edu>
Subject: Re: [LUGSB] text editors for 4+ GB files
In-Reply-To: <20041119223905.GB14521 at odd.bio.sunysb.edu>
Message-ID: <Pine.LNX.4.60.0411191746540.5384 at linux11.mnl.cs.sunysb.edu>
References: <20041119223905.GB14521 at odd.bio.sunysb.edu>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Sender: Vinay Pai <vinay at linux11>
X-BeenThere: lugsb at fsl.cs.sunysb.edu
X-Mailman-Version: 2.1.1
Precedence: list
Reply-To: Linux Users Group at Stony Brook <lugsb at fsl.cs.sunysb.edu>
List-Id: Linux Users Group at Stony Brook <lugsb.fsl.cs.sunysb.edu>
List-Unsubscribe: <http://www.fsl.cs.sunysb.edu/mailman/listinfo/lugsb>,
	<mailto:lugsb-request at fsl.cs.sunysb.edu?subject=unsubscribe>
List-Archive: <http://lists.fsl.cs.sunysb.edu/pipermail/lugsb>
List-Post: <mailto:lugsb at fsl.cs.sunysb.edu>
List-Help: <mailto:lugsb-request at fsl.cs.sunysb.edu?subject=help>
List-Subscribe: <http://www.fsl.cs.sunysb.edu/mailman/listinfo/lugsb>,
	<mailto:lugsb-request at fsl.cs.sunysb.edu?subject=subscribe>
X-List-Received-Date: Fri, 19 Nov 2004 22:49:07 -0000

What exactly are you trying to do with the file? Are you sure you really 
want a text editor?

On Fri, 19 Nov 2004, Lucas Carey wrote:

> Hi All, Can anyone recommed a good txt editor for looking at 4+ GB txt 
> files? Vim takes for ever to open up. I'd ideally like to be able to 
> open a file at a given line or at the first occurance of a given 
> pattern, ala vim. I definetly need to be able to go to the first 
> occurance of a given pattern or a given line once the file is open. I 
> have an 4-cpu alpha with 6GB of ram, so memory is not too much of an 
> issue.
>
> % wc output.file
> 113401439 547030563 5061674108
>
>
> still waiting for vim to open up to line 100600323. By this time, I've 
> already grepped fully through the file 3 times.
>
> -Lucas
> _______________________________________________
> lugsb mailing list
> lugsb at fsl.cs.sunysb.edu
> http://www.fsl.cs.sunysb.edu/mailman/listinfo/lugsb
>
>From lcarey at odd.bio.sunysb.edu Sat Nov 20 14:58:04 2004
Received: from odd.bio.sunysb.edu (odd.bio.sunysb.edu [129.49.104.25])
	iAKJw4WW018514
	for <lugsb at fsl.cs.sunysb.edu>; Sat, 20 Nov 2004 14:58:04 -0500
Received: from odd.bio.sunysb.edu (localhost [127.0.0.1])
	by odd.bio.sunysb.edu (8.12.8/8.12.8) with ESMTP id iAKJwg2Z003413
	for <lugsb at fsl.cs.sunysb.edu>; Sat, 20 Nov 2004 14:58:42 -0500
Received: (from lcarey at localhost)
	by odd.bio.sunysb.edu (8.12.8/8.12.8/Submit) id iAKJwgpA003412
	for lugsb at fsl.cs.sunysb.edu; Sat, 20 Nov 2004 14:58:42 -0500
Date: Sat, 20 Nov 2004 14:58:42 -0500
From: Lucas Carey <lcarey at odd.bio.sunysb.edu>
To: lugsb at fsl.cs.sunysb.edu
Subject: Re: [LUGSB] text editors for 4+ GB files
Message-ID: <20041120195842.GA3371 at odd.bio.sunysb.edu>
References: <200411201700.iAKH02WW016155 at filer.fsl.cs.sunysb.edu>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <200411201700.iAKH02WW016155 at filer.fsl.cs.sunysb.edu>
User-Agent: Mutt/1.4.1i
X-BeenThere: lugsb at fsl.cs.sunysb.edu
X-Mailman-Version: 2.1.1
Precedence: list
Reply-To: Linux Users Group at Stony Brook <lugsb at fsl.cs.sunysb.edu>
List-Id: Linux Users Group at Stony Brook <lugsb.fsl.cs.sunysb.edu>
List-Unsubscribe: <http://www.fsl.cs.sunysb.edu/mailman/listinfo/lugsb>,
	<mailto:lugsb-request at fsl.cs.sunysb.edu?subject=unsubscribe>
List-Archive: <http://lists.fsl.cs.sunysb.edu/pipermail/lugsb>
List-Post: <mailto:lugsb at fsl.cs.sunysb.edu>
List-Help: <mailto:lugsb-request at fsl.cs.sunysb.edu?subject=help>
List-Subscribe: <http://www.fsl.cs.sunysb.edu/mailman/listinfo/lugsb>,
	<mailto:lugsb-request at fsl.cs.sunysb.edu?subject=subscribe>
X-List-Received-Date: Sat, 20 Nov 2004 19:58:04 -0000

I guess, more accurately, I want a text reader. I have a small output file in which all results have been verified, and a larger run which is a superset of the smaller one. I want to be able to scan through the large output file and check them against the smaller one, by eye. Diff won't work due to differences in ordreing of results with similar scores. I know I could use split to break up the file, but I really shouldn't have to, seeing as how the file does fit entirely in ram. 
When I open it up in vim the RSIZE jumps to ~140MB and just sits there alternating between wait & run states, using 80-90% cpu. 
I would also like to be able to look at the results. Its a list of genes w/thier similarity scores & alignments, and I'd liked to be able to look through it for interesting genes. 
-Lucas

>What exactly are you trying to do with the file? Are you sure you really
>want a text editor?