Global Variables / System Values Decimal Places

Programming and macros
User avatar
Rob
Posts: 128
Joined: Mon Mar 08, 2021 3:46 pm
Answers: 2
Location: Mighty Glossop, UK
x 787
x 207
Contact:

Global Variables / System Values Decimal Places

Unread post by Rob »

Hey up

This has kept me 'entertained' for the last few hours.

I have some code that checks the value of a global variable against a dimension value.

The idea is that I only update the global variable if it is different.

If there are no changes then I do not save the part.

Despite initially working something changed and whatever happened my part kept saving. It had me scratching my head.

Only when I looked at all the decimal places did it dawn on me what was happening.
image.png
Now that I know the problem - I'm still not sure how to solve it.

I could check the value of the equation instead of the value, or perform rounding myself.

I guess whatever you put in 8 d.p. is the maximum you can get back from a global variable but the system value of the dimension is 13 d.p after I'd converted to mm. My naïve conversion to a string gave me 11 d.p.

Tricky stuff this programming lark.
User avatar
Rob
Posts: 128
Joined: Mon Mar 08, 2021 3:46 pm
Answers: 2
Location: Mighty Glossop, UK
x 787
x 207
Contact:

Re: Global Variables / System Values Decimal Places

Unread post by Rob »

I thinks I've solved it (well stack overflow did)
image.png
Post Reply