Harvest adding ^M to files

In case you got problems with Harvest, this link might help you

There's the full version in case you dont have a PDF viewer.

Getting rid of unwanted ^M character Posted by Ken Strobel - 2001/04/13 16:41 _____________________________________ When I add a file to Harvest from my NT platform it is showing up with a ^M character at the end of each line on my UNIX platform. I guess this is some sort of carriage return or line feed character but it renders the files useless. I'm guessing something in the file transfer to UNIX is introducing the control M character? Your thoughts are most appreciated. Ken Strobel --

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Bruce Albrecht - 2001/04/13 16:59 _____________________________________ Harvest normally stores files as binary images. When you check in a text file on Windows platforms, it saves end-of-line as carriage-return/newline (^M^J), whereas Unix systems only use newline (^J) to indicate end-of-line. If you go to the repository editor in the admin tool, you can click on the "Cross Plat..." button to enter a list of file extensions for which Harvest will convert Windows style end-of-line to Unix style end-of-line upon check-in, and vice-versa upon check-out. The versions already in the repository will still have the embedded ^Ms, however. If you check them out for update from the Unix side, and strip the ^Ms, and check them back in after adding the extensions to the Cross Platform list, then it should clear up this problem. This is covered in the Installation / Getting Started Guide in the subsection "File Conversion Issues" of the section "Additional Post-Installation Setup" of the "Installation on PC Systems" Chapter.

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Bruce Albrecht - 2001/04/16 15:13 _____________________________________ Harvest 5.0 still does end-of-line conversions by file extension. Furthermore, it's not nonsense. The only time it falls apart is if you're working with text files from multiple platforms and don't identify the extensions as needing the cross- platform conversion. Todd, if this is such a mess, what's your solution? Do you want to force the user to select end-of-line conversion on check-in/out? How do you want Harvest to guess whether the file is text or binary data?

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Karla - 2001/04/16 15:19 _____________________________________ Is there any way to use this feature for a file type that does not have an extension? I have a customer who has the ^M problem with a file named "Makefile".

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Bruce Albrecht - 2001/04/16 15:24 _____________________________________ Click on Add when the field is blank, and it will list

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Bruce Albrecht - 2001/04/16 15:28 _____________________________________ CM Crossroads - CM Crossroads FireBoard Forum Component version: 1.0.4 Generated: 28 January, 2009, 09:10Page 2

That was supposed to say (If I got the HTML right), or =less-than=NONE=greater-than.

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Todd Athorp - 2001/04/16 15:39 _____________________________________ I'll conceed maybe not complete nonsense, but still a major product lacking and a configuration/maint. pain. Harvest ought to scan for these characters at check-in, store in a common format (say just ^M for all intents and purposes), and rebuild based on the platform type. If the file was a binary, these characters wouldn't exist and the file can be checked w/out scanning. Or at least allow admins to select what file types to/to-not scan at CI/CO time. Bruce, I don't know what your exposure is to the usage of cross-platform single-source, but a lot of our developers develop in MSVC++ (but some, in the same environment may even develop new files in vi), which is then harrefreshed to solaris, nt, and eventually USS for builds. Prior to the builds we end up running dos2unix on some environments which then can screw up the date-timestamps affecting Openmake. In the 3.01b days we a tried turning on the cross-platform function for a running environment (java code I think) and the next checkin's corrupted the base deltas. The only way to restore was to delete the item from the repository then re-add. At present we turn on Cross-platform for most applications, we have not re-attempted to mess with the cross-platform extension functionality with running environments. Time permitting, I'll give this a try with 4.1.2. Todd

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Bruce Albrecht - 2001/04/16 15:58 _____________________________________ Actually, the point of a binary file is that any character may appear in the file, and that characters like ^M, ^J, or ^Z have no significance. We have several projects that primarily check code in from PCs but deliver to Sun machines, and we have had problems with the embedded ^Ms. However, our users have been educated to tell us (sometimes after the initial checkin) of any new extensions that need to be considered text files. When we know that they're going to be working from more than one platform, we try to set up all the likely extensions, and make sure they know what it means if they see a file with ^Ms in them on Unix. This sounds like a candidate for a DAR. Maybe the solution would be to have check-in/out have an option to force binary, force text conversion, otherwise use something like the Unix *file* command's database to determine if something was text or binary, and if it's a text file, try to decide if conversion was needed if the file looks like it has the Harvest agent's end-of-line format.

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Michael - 2001/04/17 01:56 _____________________________________ Hi all, first the conversion works pretty good. One exception: You use mounted disk from another OS. But this cannot work. Example: you use SAMBA or something else to access Unix-Disks from a WIN-PC or vice versa. Harvest Client on the PC does only see NT disks and convert the files. But it is an unix disk and the conversion will go the wrong way. But this is not a problem of harvest this is a problem of the SAMBA,....

=
=============================================================== CM Crossroads - CM Crossroads FireBoard Forum Component version: 1.0.4 Generated: 28 January, 2009, 09:10Page 3

Re: Getting rid of unwanted ^M character Posted by Ken Strobel - 2001/04/17 02:20 _____________________________________

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Bruce Albrecht - 2001/04/17 02:29 _____________________________________ The "." is assumed. I didn't think the documentation was clear on this particular point. If you want any file ending in .txt to be considered a text file to be converted upon check-in/out, you'll enter "txt" in the Cross-Platform extension list. It's case insensitive.

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Matt Mason - 2001/04/17 02:39 _____________________________________ Dont forget that you will need to MANUALLY remove the ^m's checked in from the PC environment BEFORE you made the change on the cross platform file types. By checking them in from the PC you basically sealed their fate so that regardless of the changes you make on the cross platform area. You have to UNDO what you have done.... Hope this helps. Matt Mason CBetter -- CBetter is a leading training and consulting facility for Harvest Users and Administrators. The next Advanced Harvest Admin class is scheduled April 24th, 25th and 26th. It will be held in (usually) sunny Santa Barbara, CA. Check out our website at HTTP://www.CBetter.com/Training

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Bruce Albrecht - 2001/04/17 02:44 _____________________________________ I think it will work if the files are checked out on the PC before adding the extensions to the cross-platform extensions list, and then checked back in after adding the extension. However, if you do the stripping of the ^M's from the Unix system, it doesn't matter when you add the extension to the list.

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Ken Strobel - 2001/04/17 02:51 _____________________________________ Thank you so much! That solved the problem.

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Todd Athorp - 2001/04/17 02:53 _____________________________________ CM Crossroads - CM Crossroads FireBoard Forum Component version: 1.0.4 Generated: 28 January, 2009, 09:10Page 4

Is this cross-platform nonsense removed in 5.0? This has been a real stumbling point for a number of companies, us included. With some applications moving towards single-source running on multiple-platforms, this concern will not be going aways and will become much larger over time. While what's been talked about in the earlier post is yet another workaround for a 3.x/4.x limitation, this needs to be addressed natively inside Harvest going forward! -- Thanks, Todd Athorp 414-577-9448 Senior Programmer Analyst Metavante Corporation - Enterprise Configuration Services Current Harvest Version: 4.1.2 Current Openmake Version: 5.11 http://www.execpc.com/antilles http://www.metavante.com

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Deborah - 2001/04/17 11:38 _____________________________________ Are we the only folks who see these ^M's at random, even though we have the Cross Platform list set up correctly? It works most of the time but not all of the time. Tech support says that they are just using ftp and that they use the Cross Platform list to set bin or ASCII for ftp. Hence the problem is in our FTP software. However, we cannot make it happen moving stuff around with ftp alone. Our server and repositories are on a Unix machine and most of our users are using NT clients. If we were checking out to one system, ftping to the other incorrectly, and checking in from the other, we might see this but I think I can guarantee that no one is doing this. Deborah Rector

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Ron Rouhof - 2001/04/17 12:54 _____________________________________ At one of our customers we also see this problem. We think that it is someting on the local settings of the machine on wich the user is running. We only see it at some specific machines and not on all. We still have not found what the exact problem is?? Ron Rouhof

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Joe - 2001/04/17 21:12 _____________________________________ Hello folks, Our Company also has had a recurring problem with the end line characters difference between Windows and Unix - this was a problem in CVS, in CMVC, in TeamConnect and now in Harvest. Our situation is now further complicated by the fact that Sun Packages ( not to be confused with Harvest Packages) must be handled as a text file, even if they contain binaries within them. Since our company is moving to standardize on Sun Packages for releasing to production and test environments, the ability to recognise how to handle specific types of files is more "interesting"...In the past we on the CMT would handle the ^M issue by scrubbing the files ourselves using vi and then have the "guilty" developers buy us lunch - after a time or two at the buffet line, they got smart and scrubbed their own files before checking them in...I CM Crossroads - CM Crossroads FireBoard Forum Component version: 1.0.4 Generated: 28 January, 2009, 09:10Page 5

know it won't work for everyone, but hey - it worked for us and we got fed to do it...Seriously, though, knowing what type of file is being handled is very important, but I can't see a programatic way to handle it, since for every method we have thought of for identifying a file type, we have come up with a possible exception - it comes to the developers and the CM teams working together and knowing each other's worlds and limitations of the tools being used...- Kind of preachy, isn't it?? If anyone has a fool proof way to solve this, I am all ears... Thanks, Joe

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by brooke - 2001/04/18 08:54 _____________________________________ Joe, Do you have cross-platform set up properly? We handle control characters like we handle viruses (viri?) that developers/testers/tech writers might check in with their word docs: it is their responsibility to ensure they put "clean" files in the bank. We use a UDP pre/post to remind them to run a virus check. We haven't had any problems with ^M chars with X-platform turned on. You guys are using Sun packages? Are you still an affilliate of BellSouth? Why aren't you guys using DDS? Brooke.Gravitt1@bellsouth.com

=
=============================================================== Re: Getting rid of unwanted ^M character Posted by Ken Strobel - 2001/04/19 02:35 _____________________________________ My thanks to Bruce Albrecht for clueing me in to the Cross platform text file extensions in the Repository Editor. This provided me with the SELECTIVE file conversion that I need. -Ken Strobel

=
=============================================================== CM Crossroads - CM Crossroads FireBoard Forum Component version: 1.0.4 Generated: 28 January, 2009, 09:10