Results 1 to 1 of 1

Thread: Ubuntu, perl and utf-8 character enconding

  1. #1
    Join Date
    Jun 2008
    Beans
    11
    Distro
    Ubuntu 8.04 Hardy Heron

    Ubuntu, perl and utf-8 character enconding

    Beginner in Perl.

    I made a Perl script that parse data from html site. My script encodes the parse data in UTF-8, one of the data contains romanian characters, so encoding the data results in incorrect characters such as:
    (befor and after I set default environment (Ubuntu 10.04) to use the Romanian code page)

    ţ = (incorrect); ş = (incorrect); ă = Ń (correct);

    line example to parse from html (without html tags):

    Distribuţia: Robert Downey Jr. (Sherlock Holmes) Jude Law (Dr. John Watson) Rachel McAdams (Irene Adler) Mark Strong (Lord Blackwood) Kelly Reilly (Mary Morstan) Eddie Marsan (Inspectorul Lestrade) James Fox (Sir Thomas)

    I want to split this with:

    Code:
    my ($credits, $line)
    foreach $credits (split /(?=\w+:)\s*/, $line) {
    ...
    Output, because ţ is encoded incorrect as "", this is interpreted as "non-word character", line breaks incorrectly here, is:

    Distribu■
    Robert Downey Jr. (Sherlock Holmes)
    Jude Law (Dr. John Watson)
    Rachel McAdams (Irene Adler)
    Mark Strong (Lord Blackwood)
    Kelly Reilly (Mary Morstan)
    Eddie Marsan (Inspectorul Lestrade)
    James Fox (Sir Thomas)

    Output wanted (correct):

    Distribuţia
    Robert Downey Jr. (Sherlock Holmes)
    Jude Law (Dr. John Watson)
    Rachel McAdams (Irene Adler)
    Mark Strong (Lord Blackwood)
    Kelly Reilly (Mary Morstan)
    Eddie Marsan (Inspectorul Lestrade)
    James Fox (Sir Thomas)

    If I use "\p{Alpha}" variable instead of "\w", partially solve the problem (line breaks correctly, but displays the "Distribu■ia" rather than "Distribuţia", look like this (incorrect):

    Distribu■ia
    Robert Downey Jr. (Sherlock Holmes)
    Jude Law (Dr. John Watson)
    Rachel McAdams (Irene Adler)
    Mark Strong (Lord Blackwood)
    Kelly Reilly (Mary Morstan)
    Eddie Marsan (Inspectorul Lestrade)
    James Fox (Sir Thomas)
    Last edited by aaalbatrosss; September 13th, 2011 at 11:57 AM.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •