Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

string.gmatch eating framerate

edited July 2015 in Questions Posts: 7

Alright so im making a little program that can read xml files, which i use string.gmatch to filter out all the information i need. But when i run huge files trough the loop, my framerate drops all the way to 0
I can't really show any functional code, since its for something i will get payed for.
I just wonder if theres any way to make it not eat my framerate and/or if this is a known issue.

Example of the loop im using:

for k,b in string.gmatch(SomeStringHere, SomePattern) do 
   -- do stuff here
end

EDIT: got it fixed, used a different way which is way quicker

Still dont know what caused the problem though

Comments

  • IgnatzIgnatz Mod
    Posts: 5,396
    1. How many gmatches can you do before t gets slow?

    2. Do you need to do all the gmatches in one frame, or can you spread them across several frames, eg if you only want a 3 second response time, you can spread your searches across 180 frames.

  • Posts: 2,020

    How come you need to read an xml file every frame? Why not just read it once, parse the information into something that's very quick to access, like a table, and access that every frame?

  • Posts: 7

    @Ignatz @yojimbo2000 No, no. My bad. Im not calling it every frame, just once at the start.

  • edited July 2015 Posts: 2,020

    How impatient are the users? Either put up a "please wait a moment" sign, or place the gmatch in a coroutine to give the user a gradually updating status bar (though that will almost certainly slow it down even further, eg if yield is inside the gmatch loop then each iteration would take at least a frame, so I wouldn't recommend that). Can you at least show us the pattern in the gmatch? How many iterations are you getting from these huge files?

  • dave1707dave1707 Mod
    Posts: 7,682

    @Luxizzle How big are the files you're using. I have a 3,151,541 character dictionary file (300249 words) that I use. If I do a string.gmatch on the last word, it reads the whole file in the blink of an eye. What are you doing with the results of gmatch, maybe that's what's slowing things down.

  • Posts: 2,020

    Anchoring the search with ^ and $ helps a lot.

  • dave1707dave1707 Mod
    Posts: 7,682

    @Luxizzle Since you don't want to show any code or say what you're doing, I'll take a guess that you want to view selected information from an xml file. Here's an example I have that reads an html file, but xml is probably the same. I have this example setup to show any text between the < and > characters in the file. You can drag your finger up or down the screen to see all the lines. Let me know if this is usable or not close to what you want.

    displayMode(FULLSCREEN)
    supportedOrientations(LANDSCAPE_ANY)
    
    function setup()
        font("Courier")
        textMode(CORNER)
        st=1
        str=""
        tab={}
        dy=0
        url="https://github.com/cbracco/html5-test-page/blob/master/test.html"    
        http.request(url,response)
        msg="Please wait, reading file"
    end
    
    function response(data,status,headers)
        for a,b,c in string.gmatch(data,"(<)(.-)(>)") do
            table.insert(tab,b)
        end
    end
    
    function draw()
        background(40, 40, 50)
        fill(255)
        if #tab==0 then
            text(msg,350,350)
        else
            st=math.ceil(dy/20)+1
            en=math.min(st+50,#tab)
            line=0
            for z=st,en do
                line=line+1
                str=string.format("%4d %s",z,string.sub(tab[z],1,90))
                text(str,0,HEIGHT-line*25)
            end
        end
    end
    
    function touched(t)
        if t.state==MOVING then
            dy=math.max(dy+t.deltaY,0)
        end
    end
    
Sign In or Register to comment.